Running the Checks

Every page in this section ends with a Check yourself — a short experiment that either confirms the page’s claim or breaks it. They are all the same kind of thing: a few lines of Python, run against a real but small model that fits on a laptop. What none of them spell out, because it would mean saying it thirty times, is the part that comes before the experiment — which language, what to install, how the model gets loaded, and what the handful of objects the checks keep naming actually are. This page is that missing preamble. Read it once and keep it open; after that, every Check yourself is copy-and-run.

The three kinds of check#

The experiments differ in what they need in front of you:

Everything below is Python 3. The rest of this page sets up each of the three in turn.

Load and look: GPT-2 small in HuggingFace#

Two packages cover every load-and-look check:

1

pip install torch transformers

torch is PyTorch, the array-and-autograd library the model’s arithmetic runs on; transformers is HuggingFace’s library of pretrained models, which is where GPT-2 comes from. With those installed, the following is the setup that every load-and-look Check yourself silently assumes:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

import torch
from transformers import GPT2LMHeadModel, GPT2TokenizerFast

# "gpt2" is the 124M-parameter small model — downloaded once, then cached on disk.
model = GPT2LMHeadModel.from_pretrained("gpt2")
model.eval()   # turn off dropout, so two runs of the same input give the same answer
tok = GPT2TokenizerFast.from_pretrained("gpt2")

# Tokenize a prompt. input_ids is a tensor of shape [1, n]: one sequence, n token ids.
ids = tok("The cat sat on the", return_tensors="pt").input_ids

with torch.no_grad():                     # we aren't training, so skip gradient bookkeeping
    outputs = model(ids, labels=ids, output_hidden_states=True)

That one forward pass produces everything the load-and-look checks read. The three keyword arguments and the objects they fill are the vocabulary those checks are written in:

  • labels=ids makes the model score itself. It returns outputs.loss, the average cross-entropy over the sequence. You pass the same tensor as both input and labels because there is no separate label — the model shifts the text by one internally and predicts each token from the ones before it. Exponentiate the loss and you have perplexity: torch.exp(outputs.loss).
  • output_hidden_states=True fills outputs.hidden_states, a tuple of 13 tensors each shaped [1, n, 768] — the embedding output, then one snapshot after each of the 12 blocks. hidden_states[0] is the input before any block has run; hidden_states[-1] is already past the final normalization, which HuggingFace applies before handing back that last entry. (Left off, outputs.hidden_states is None.)
  • outputs.logits, always present, is shaped [1, n, 50257]: one score for every token in the vocabulary, at every position.

The named pieces of the model that the checks reach into all hang off model:

1
2
3
4
5
6
7

model.transformer.wte.weight     token embedding table, 50257 × 768 — also the unembedding (tied)
model.transformer.wpe.weight     position embedding table, 1024 × 768
model.transformer.h[i]           block i (i = 0…11)
model.transformer.h[i].attn.c_attn   fused Q/K/V projection, 768 × 2304
model.transformer.h[i].attn.c_proj   attention's output projection — the hook point below
model.transformer.h[i].mlp           the block's MLP
model.lm_head.weight             the unembedding — the same storage as wte.weight

That wte/lm_head sharing is weight tying, and c_attn being one matrix rather than three is covered under Q/K/V projections.

Two more switches some checks flip#

  • Attention weights. To read the attention pattern itself, load with attn_implementation="eager" and pass output_attentions=True; then outputs.attentions is a tuple of [1, 12, n, n] matrices. The faster default SDPA (scaled dot-product attention) path doesn’t reliably hand those back, so the checks in one attention head and attention pin eager on purpose.
  • The KV cache. model.generate(...) and a plain forward pass both take use_cache=True (the default) or False. The tokens are identical either way — the KV cache is an optimization, not an approximation — which is exactly the check on that page.

Replacing a module’s output mid-pass#

The attention and residual stream checks delete part of the model while it runs, using a forward hook — PyTorch’s callback that fires when a module produces its output and can return a replacement:

1
2
3
4
5
6

def zero_out(module, inputs, output):
    return torch.zeros_like(output)

handle = model.transformer.h[6].attn.c_proj.register_forward_hook(zero_out)
# ... run the model; block 6's attention now contributes nothing ...
handle.remove()   # undo it

Hook c_proj rather than .attn itself: .attn returns a tuple, not a plain tensor, and the block would index into your replacement instead of using it.

Generating text#

The sampling and decoding checks call model.generate(...). Greedy decoding is do_sample=False; nucleus sampling is do_sample=True, top_p=0.9; speculative decoding passes a small assistant_model=. See sampling strategies for what the arguments do.

A few checks (RLHF, multi-head attention) swap "gpt2" for another HuggingFace model id — "gpt2-xl", "Qwen/Qwen2.5-0.5B" — but change nothing else about the setup above.

Train a tiny model: nanoGPT#

The architecture checks need to modify the model and retrain, which the HuggingFace weights don’t let you do cleanly. They use nanoGPT — Andrej Karpathy’s roughly 300-line implementation of this exact model, small enough to read end to end. Clone it and install its dependencies:

1
2
3

git clone https://github.com/karpathy/nanoGPT
cd nanoGPT
pip install torch numpy transformers datasets tiktoken wandb tqdm

The quickstart trains a character-level model on a 1 MB Shakespeare corpus in minutes on a CPU:

1
2

python data/shakespeare_char/prepare.py                    # build the dataset
python train.py config/train_shakespeare_char.py --device=cpu --compile=False

Two files matter for the checks. model.py holds the architecture — this is where “in nanoGPT, delete both skips” or “swap one block’s MLP for eight copies plus a router” happen, as edits to those ~300 lines. train.py runs the training loop and prints the loss you’re asked to watch. nanoGPT can also load GPT-2’s pretrained weights, with GPT.from_pretrained('gpt2'), which is how the grouped-query attention check starts from the real model before changing it.

No model: just torch (or tiktoken)#

Some checks download nothing at all. The ones on GELU, RoPE, superposition, and context length are arithmetic on random tensors — pip install torch and you have everything, no weights involved. Build a random matrix with torch.randn(768, 3072), call torch.nn.functional.gelu, and you’re running the experiment.

The tokenization check needs only the GPT-2 tokenizer, which ships without the model in a separate small package:

1

pip install tiktoken
1
2
3

import tiktoken
enc = tiktoken.get_encoding("gpt2")
enc.encode(" cat")   # -> [3797]

Check yourself#

The harness has its own falsifiable check: run the load-and-look setup at the top of this page verbatim, then confirm it’s wired up before you trust any experiment built on it.

1
2
3

len(outputs.hidden_states)          # 13 — embedding plus one per block
outputs.logits.shape                # torch.Size([1, 5, 50257])
torch.exp(outputs.loss)             # GPT-2's perplexity on the sentence

hidden_states should have 13 entries; if it has none, you dropped output_hidden_states=True. The logits’ last axis is the 50,257-token vocabulary. And torch.exp(outputs.loss) comes back a two-digit number for ordinary English — the same quantity the perplexity page opens on. If outputs.loss is missing entirely, you dropped labels=ids.

Depends on / leads to#

Depends on GPT-2, the model these checks all run against. Leads to every page’s Check yourself; the reading order proper resumes at tokenization.

Backward GPT-2 Tokenization Forward