• Week 12 — the QKV abstraction, scaled dot-product attention, self-attention. We stopped at multi-head as a preview.

• Weeks 13–14 — your presentations: CNNs, ResNets, double descent, RNNs/LSTMs, transfer learning, scaling laws.

This week we close the loop:

• Build the architecture behind every frontier LLM.
• Train it the way the field actually trains it.
• End on what is open right now.

• A function from a sequence of tokens to a probability distribution over the next token.

• The vocabulary \(\mathcal{V}\) has size \(V\) (typically 32k–128k).

• The output lives on the simplex \(\Delta^{V-1}\) — a probability vector over \(V\) choices.

• The parameters are estimated by maximum likelihood.

The natural move — assign each word an integer ID — has issues:

• The integer is arbitrary: “cat” might be 742, “kitten” might be 5811.

• The IDs say nothing about meaning. “742” doesn’t tell you “cat” is similar to “kitten.”

• Arithmetic on IDs is meaningless. \(5811 - 742 = 5069\) is not a feature of language.

We need something better than an arbitrary label. But first: what is the unit we should be labeling?

• Frequent words → a single token.
• Rare words → split into pieces.
• The vocabulary is fixed before training. Each token has a unique integer ID.

The most common scheme is Byte Pair Encoding (BPE): start with characters, iteratively merge the most frequent adjacent pair until the vocabulary reaches the target size.

• Character level (\(V \approx 100\)): tiny vocabulary, but every word becomes a long sequence. Length explodes.

• Word level (\(V \approx 10^6\)): short sequences, but rare words become “unknown” and the embedding matrix is huge.

• Subword sits in between: balanced.

• “cat” and “kitten” are near each other.

• “cat” and “thermodynamics” are far.

• Relationships like “kings vs queens” mirror “men vs women.”

We never engineer this geometry. The model learns it from data.

• Define a matrix \(\mathbf{E} \in \mathbb{R}^{V \times d}\).

• Row \(v\) holds the \(d\)-dimensional vector for token \(v\).

• Whenever the input contains token \(v\), replace its integer ID with that row.

• These vectors are parameters — estimated jointly with the rest of the model by MLE.

Typical \(d\):

• \(768\) for small models (GPT-2)
• \(4{,}096\) for \(8\)B-scale (Llama 3 8B)
• \(12{,}288\) for the largest production models

Yet after training, vectors organize themselves:

• Synonyms cluster.
• Words from the same domain (medical, legal, code) form regions.
• Analogies become arithmetic.

The famous example:

\[\mathbf{e}_{\text{king}} - \mathbf{e}_{\text{man}} + \mathbf{e}_{\text{woman}} \approx \mathbf{e}_{\text{queen}}.\]

This is the distributional hypothesis in vector form: tokens used in similar contexts end up nearby in \(\mathbb{R}^d\).

• We took a categorical predictor with \(V\) levels.

• We replaced one-hot encoding with a learned dense representation in \(\mathbb{R}^d\).

• The representation is estimated jointly with the prediction task.

• Permute the rows of \(\mathbf{H}\): the output rows permute identically.

• Nothing else changes. Same attention weights, same content.

• The model has no way to tell that “The cat sat on the mat” differs from “the mat sat on the cat.”

This is fine when the input really is a set (like a bag of objects). For language — where order is the whole point — it is broken.

Three common strategies:

• Sinusoidal (original Transformer): add a fixed pattern \(\mathbf{p}_t\) of sines and cosines at multiple frequencies. Like a Fourier basis for position.

• Learned (BERT, GPT-2): treat each position as a categorical index and learn \(\mathbf{p}_t\) as a parameter.

• Rotary (RoPE) (Llama, Mistral, Qwen): rotate query and key vectors by an angle that depends on position — inside the attention computation, not at the input.

What matters: every token’s vector is now a function of what it is and where it is. The attention mechanism then has the information it needs to use position.

Then build that position’s new representation as a blend of every other position’s representation, weighted by how much each one matters.

That is the entire mechanism.

Three things we need to make this concrete:

• A way to ask the question.
• A way for each position to advertise what it offers.
• A way to combine the answers.

Three learned linear maps create three views of the same tokens:

• \(\mathbf{Q} = \mathbf{H}\mathbf{W}^Q\) — what each token asks.

• \(\mathbf{K} = \mathbf{H}\mathbf{W}^K\) — what each token advertises.

• \(\mathbf{V} = \mathbf{H}\mathbf{W}^V\) — what each token carries (the payload).

The projection matrices \(\mathbf{W}^Q, \mathbf{W}^K, \mathbf{W}^V\) are learned — the model decides what “asking,” “advertising,” and “carrying” mean during training.

What this number means:

• Big and positive → the vectors point similarly → high relevance.
• Near zero → vectors are perpendicular → no relevance.
• Big and negative → vectors oppose each other → anti-relevance.

For each query, we get \(T\) scores — one per token in the sequence.

Pause on that shape…

But the relationships between tokens are heterogeneous:

• Syntactic — subject of the verb.
• Semantic — which noun does “it” refer to.
• Positional — adjacency, sequence order.

Forcing one set of \((\mathbf{W}^Q, \mathbf{W}^K, \mathbf{W}^V)\) to capture all of these is asking too much.

• Each head \(i\) has its own \((\mathbf{W}^Q_i, \mathbf{W}^K_i, \mathbf{W}^V_i)\).

• Each operates in its own \(d/h\)-dimensional subspace.

• The results are concatenated and mixed by a final output projection \(\mathbf{W}^O\).

Setting \(d_k = d/h\) keeps the total compute the same. More expressiveness, no extra cost.

• Some attend to the previous token.

• Some attend to the next occurrence of the same token.

• Some track syntactic dependencies (subject of the current verb).

• Some appear to do nothing identifiable.

A caveat: this is an empirical phenomenon, not an architectural guarantee. The model is free to use the heads however it likes — as long as the loss goes down.

When predicting token \(x_{t+1}\):

• The model must use only positions \(1, 2, \ldots, t\).

• Never \(t+1\), never the future.

• Otherwise it would be “cheating”: reading the answer before predicting it.

But standard self-attention happily attends in both directions. Position \(t\) would see \(t+1, t+2, \ldots, T\).

We need to break that.

• Before the softmax, set the upper-triangular entries of \(\mathbf{Q}\mathbf{K}^\top / \sqrt{d_k}\) to \(-\infty\).

• After softmax, those entries become zero.

• Each row sums to one over the past and present only.

• The attention weight matrix is now lower triangular.

That is the entire change. Multi-head, FFN, residuals, layer norm — all unchanged.

The model’s joint distribution over a sequence factorizes:

\[P_{\boldsymbol{\theta}}(x_1, \ldots, x_T) = \prod_{t=1}^{T} P_{\boldsymbol{\theta}}(x_t \mid x_1, \ldots, x_{t-1}).\]

Wednesday:

• Stack everything into a transformer block.
• See how it is trained (just MLE).
• See how it is used (sampling).
• End on the open frontier.

• Tokens → embeddings → positions → attention.

• The QKV pipeline is Nadaraya–Watson with a learned kernel.

• Multi-head: an ensemble of those kernel regressions.

• Causal mask: enforces the autoregressive factorization.

Today: how do we wrap this into a model, train it, and use it?

After the final block, project each token’s \(d\)-vector to vocabulary logits and softmax.

After 2022, the field consolidated around decoder-only. Why?

• A single decoder-only model trained on enough text can do everything the others were designed for.

• Simplest of the three: one input format, one output format, one training objective.

• Simplicity scales. Fewer choices to tune as you increase \(N\) and \(D\).

Every frontier LLM in 2026 is decoder-only.

Where the 12 comes from:

• 4 attention matrices (\(\mathbf{W}^Q, \mathbf{W}^K, \mathbf{W}^V, \mathbf{W}^O\)): \(4d^2\).
• 2 FFN matrices of size \(d \times 4d\): \(8d^2\).
• Total per block: \(12 d^2\).

• \(\sim 2N\) FLOPs per token forward pass.
• \(\sim 4N\) FLOPs per token backward pass.

• Feed the model a long sequence of tokens.

• At every position, ask: what is the probability the model assigns to the actual next token?

• Adjust parameters so that probability goes up.

• Repeat over \(\sim 10^{13}\) tokens.

That is it. The objective is the average negative log-likelihood of the next token, summed over every position of every document in the corpus.

• Week 6. Logistic regression: minimize Bernoulli NLL. \(-\sum [y \log \hat{p} + (1-y)\log(1-\hat{p})]\).

• Week 11. Neural network classification: minimize multinomial NLL on softmax outputs.

• Today. LLM pre-training: minimize multinomial NLL on softmax outputs of a \(V\)-way conditional categorical, at every position.

• Pre-training does the heavy lifting. SFT and RLHF use less than 1% of total compute.

• After pre-training: a fluent base model that does not yet “follow instructions.”

• After SFT: instruction-following.

• After RLHF: aligned with human preferences (helpfulness, honesty, safety).

• Some qualities are easy to recognize but hard to demonstrate: “this answer is more helpful,” “this one is less misleading.”

• Human raters can compare two answers reliably, even when they cannot write a perfect one.

• So: collect pairwise comparisons, learn from those.

For each prompt \(x\), sample two responses \(y\) and \(y'\). Ask a human: which is better?

We need a statistical model that turns pairwise comparisons into a ranking score for each response.

What this says:

• Each response has a latent score \(r\).

• The probability \(y\) beats \(y'\) is the logistic function of the score difference.

• Greedy. Take \(\arg\max_v p_v^{(t)}\). Deterministic. Often repetitive.

• Temperature \(\tau\). Rescale logits by \(1/\tau\) before softmax. \(\tau < 1\) sharpens (more deterministic). \(\tau > 1\) flattens (more random). Same idea as Boltzmann distributions.

• Top-\(k\) / top-\(p\) (nucleus). Truncate the distribution: keep only the top \(k\) tokens, or the smallest set whose probability exceeds \(p\). Then renormalize and sample.

Statistically, all of these are just modifications of the conditional distribution being sampled at each step — restricted-support sampling on a categorical.

Typical settings: \(\tau = 0.7\), top-\(p = 0.9\), top-\(k = 50\).

The Chinchilla paper (Hoffmann et al., 2022) fit them jointly:

\[L(N, D) = L_\infty + \frac{A}{N^{\alpha}} + \frac{B}{D^{\beta}}.\]

With \(\alpha \approx 0.34\), \(\beta \approx 0.28\), \(L_\infty \approx 1.69\).

• \(L_\infty\) — the irreducible loss. The entropy of the data itself.

• \(A / N^\alpha\) — shrinks as the model grows. Finite-model bias.

• \(B / D^\beta\) — shrinks as data grows. Finite-data variance.

Five live areas, each with statistical content:

• Data mixture. Pre-training data is a mixture over sources (web, code, books). The optimal mix is unknown and scale-dependent.

• Reasoning and inference-time compute. Models that “think” before answering — a new axis in the scaling-law picture.

• Multimodality. Images, audio, video tokenized into the same space as text.

• Agents and tool use. Long-horizon decision-making with reinforcement learning at the loop level.

• Uncertainty quantification. Calibrated uncertainty on LLM outputs.

• Bias–variance. Week 1’s polynomial fitting → Week 11’s neural networks → today’s Chinchilla law. The form \(L = L_\infty + A/N^\alpha + B/D^\beta\) is bias–variance for the modern regime.

• Maximum likelihood. Week 6’s logistic regression → today’s LLM pre-training. The model class changed. The loss did not.

• Regularization. Week 3’s ridge → Week 12’s weight decay → LLM pre-training’s weight decay. Early stopping, dropout, the implicit regularization of SGD — the toolkit migrated, the principle stayed.