Table of Contents

AI Foundations

AI vs ML vs Deep Learning vs Generative AI

These are nested, not parallel — think concentric circles, not a Venn diagram.

AI — the umbrella. Any system that performs tasks we’d call “intelligent” if a human did them. Includes rule-based expert systems, search algorithms, even a chess engine with hand-coded heuristics. No learning required.

ML — a subset of AI where the system learns patterns from data instead of being explicitly programmed with rules. Your classic logistic regression, random forest, XGBoost model — all ML, none of it “deep.”

Deep Learning — a subset of ML using neural networks with many layers (“deep” = depth of layers). The shift DL brought: instead of you hand-engineering features (e.g., “extract edge gradients from this image”), the network learns the feature representations itself, layer by layer. This is why DL exploded once GPUs made training deep stacks feasible — it scales with data and compute in a way classical ML doesn’t.

Generative AI — a subset of DL focused on generating new content (text, images, audio, code) rather than just classifying or predicting a number. LLMs, diffusion models (image gen), GANs — all generative. The architectural breakthrough that made modern GenAI viable for text was the transformer (2017), which is the backbone of every LLM you use today.

So: GenAI ⊂ Deep Learning ⊂ ML ⊂ AI. An LLM is “AI,” but calling a 1990s expert system “generative AI” would be wrong.

How LLMs Work — Tokens, Embeddings, Attention

Tokens. Text isn’t fed in as words — it’s broken into tokens, which are often sub-word chunks (“unbelievable” might become “un” + “believ” + “able”). This lets the model handle rare words, typos, and multiple languages without an infinite vocabulary. A token is the atomic unit the model actually reasons over.

Embeddings. Each token gets mapped to a vector — a list of numbers (think 1,000+ dimensions) that represents its “meaning” in a geometric space. Words used in similar contexts end up with similar vectors. This is the same embedding concept you’d recognize from recommendation systems or search — just scaled up and learned end-to-end with the rest of the model.

Attention. This is the actual innovation. For every token, the model asks: “given all the other tokens in this context, which ones matter most for understanding me?” It computes a weighted relevance score across the entire input and blends information accordingly. This is why LLMs handle long-range dependencies well — “it” in sentence 10 can attend directly back to a noun in sentence 1, no sequential bottleneck like older RNN/LSTM architectures had.

Stack many layers of attention + feedforward transformations on top of each other, scale to billions of parameters, train on massive text corpora — that’s a transformer-based LLM. The model’s entire “skill” is next-token prediction; everything that looks like reasoning, coding ability, or knowledge emerges from doing that one task extremely well at scale.

Training vs Inference — Why Inference Cost Dominates Production

Training is a one-time (or periodic) cost: feed the model massive data, adjust billions of parameters via backpropagation, run for weeks on thousands of GPUs. Expensive, but it’s a capex event — you pay it once per model version.

Inference is what happens every single time a user sends a prompt and the model generates a response. This is opex — it scales linearly with usage, forever.

Why inference dominates the real cost conversation in production systems:

Volume. Training happens once; inference happens millions/billions of times across a product’s life. Even if a single inference call is cheap, multiply by request volume and it eclipses training cost within months for any product with real traffic.
Latency matters in production, accuracy in research. Training optimizes for the best possible model. Inference optimizes for serving that model fast and cheap at scale — this is where techniques like quantization, KV-caching, batching, and speculative decoding live. As an architect, this is the layer you actually design around.
Autoregressive generation is inherently sequential. Generating each output token requires a forward pass through the whole model, and each new token depends on all previous ones. You can’t fully parallelize generation the way you can parallelize training across a batch. This sequential nature is exactly why inference optimization is its own discipline now — vLLM, TensorRT-LLM, continuous batching, etc.
Cost-per-token economics drive architecture decisions. Whether you self-host an open model on your own GPUs vs. call a hosted API vs. use a smaller fine-tuned model for a narrow task — that decision tree is entirely an inference-cost optimization, not a training one. This is the conversation you’re already having in AWS architecture contexts: it’s directly analogous to compute provisioning decisions (on-demand vs. reserved vs. spot), just at the model-serving layer instead of the EC2 layer.

AI Foundation Models — What “Foundation” Actually Means

An AI foundation model is a model trained on broad, general-purpose data at massive scale, designed to be a base layer that gets adapted to many downstream tasks rather than built for one narrow purpose.

The term (coined by Stanford’s HAI in 2021) captures three properties:

Generality — trained on broad data (web text, code, images), not a narrow labeled dataset for one task. GPT-4, Claude, Llama, Gemini — all foundation models.
Adaptability — instead of training a new model from scratch for every use case, you adapt the foundation model via prompting, fine-tuning, or RAG. This is the architectural shift: the foundation model is infrastructure you build on top of, the same way you’d treat a cloud platform as infrastructure rather than building your own data center per project.
Emergent capability at scale — abilities that weren’t explicitly trained for (multi-step reasoning, code generation from natural language) show up once the model crosses certain scale thresholds. This is still not fully predictable in advance, which is part of why evaluation is its own active subfield.

The architectural implication for you as an AWS/Java architect: foundation models change where the “build vs buy” line sits. Instead of architecting a custom ML pipeline per problem (data collection → feature engineering → training → deployment), you’re now architecting around a pre-trained foundation model plus a thin adaptation layer (prompt engineering, RAG, fine-tuning) — which shifts the system design conversation from “how do we train this” to “how do we integrate, ground, and govern this.”