How models scaled up
How big is 'big', really?
The idea inside
The same architecture scaled by orders of magnitude.
After this lesson
You can explain that frontier models are the same architecture, just scaled enormously.
Where it leads
Scaling the model is one axis. But there's another: let it think longer at answer-time.
Inside this lesson
That's the real lesson stage, paused. Claim your pass to operate it.
See how AI actually works, end to end.
This lesson is one stop on the full arc. Unlock all of it, and keep it for life.
What you get
- The 34-lesson main path, a finishable route from a word to agents
- Goal tracks for using AI at work and building AI features
- Boss labs that make you apply a whole act, not just recognize it
- Spaced recall that brings each idea back before you forget
- Course memory: every term defined, with links to where it first appears
- A shareable capability card when you finish the main path
- Lifetime access on every device, every future lesson included
Not videos to watch. You predict, operate the machine, then prove it. That is why it stays.
99 interactive lessons and challenges. No videos, no code.
Free launch pass: lifetime access, no card needed
New here? The first lessons are free to try. Start with lesson 0.1
What this lesson shows
The same architecture scaled by orders of magnitude.
The question it opens with
How big is 'big', really?
The walkthrough, in the lesson's own words
- Same machine. Orders of magnitude more of it.
- GPT-2 is the speck on the left. Drag the slider to a frontier model.
- Here's the jump, at true scale. GPT-2 is a speck; the frontier runs off the edge.
- Your turn: scrub through the years. Toggle the scale to find the small ones.
- GPT-2, the transformer you built in Act 3, had 1.5 billion parameters. How much bigger is a 2026 frontier model? Drag and find out.
- Roughly 2,000× the parameters of GPT-2: the same transformer, vastly scaled.
- One dip ahead, and it's on purpose: Chinchilla is smaller because data can matter more than size (lesson 4.4).
- More parameters = the same transformer blocks from Act 3, just deeper stacks and wider embeddings.
- True scale: GPT-2 is a speck, the frontier runs off the edge.
- Log scale squeezes huge gaps so every model stays on screen.
- The baseline, every other model is measured against this.
- Not a typo. Chinchilla is smaller on purpose: lesson 4.4 showed it's often better to add DATA than just parameters.
- Exhibit · approximate public figures, not a live model.
- It's the very same transformer blocks from Act 3, just stacked deeper and wider. No new ideas. Only more.
- 1.5B to trillions of parameters: no new ideas, just more. You already know the machine.
- Scale is the headline, not the whole story. Modern assistants also add post-training and preference tuning (2.6), reasoning modes that spend extra tokens thinking (4.8), tools and agent loops (Acts 5 and 6), and images and audio via multimodality (1.5). Same core machine, with new training and new plumbing around it.
- A newer model handles a tricky request that an older one flubbed, even though both are just predicting the next token. If the machine is the same, what actually changed?
- From GPT-2 to today's frontier models it's the same transformer, just scaled by orders of magnitude in parameters, data, and context. That extra scale is what unlocked new capabilities, so when a task needs more, often the practical fix is reaching for a bigger, newer model rather than a different kind of machine.
Key takeaway
It's the same machine, vastly scaled.
What you can do after this lesson
You can explain that frontier models are the same architecture, just scaled enormously.
Where it leads: Scaling the model is one axis. But there's another: let it think longer at answer-time.
This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.