Why GPUs beat CPUs

What this lesson shows

Each output cell of a layer is its own independent sum, so all of them can be computed at once.

The question it opens with

A giant matrix multiply on a few fast CPU cores takes forever.

The walkthrough, in the lesson's own words

• Same 36 cells, two machines. Tap the one you'd back, then Fire.
• Why can a GPU do all 36 at once? Each cell is its own little sum.
• Right, no cell needs another's answer first, so the GPU does all 36 at once and wins by a mile.
• The CPU's few workers must take the cells in rounds. The GPU has far more, so it grabs all 36 at once. Wide wins here.
• Each cell uses its own weight row (lesson 3.6) and nothing else. Rows don't wait on each other, so every cell can be computed at the same time. Each GPU worker is a bit slower than a CPU one, but there are so many that doing all 36 in a single shot still wins.
• Because the work is parallel, wide hardware wins: a GPU does every cell in one shot. That's why AI runs on GPUs.
• Attention (lesson 3.2) is the same kind of math, big batches of multiply-and-sums between words, so the whole transformer runs wide on a GPU.
• A GPU is faster than a CPU, or it's just the graphics chip, so that's why AI needs one.
• Each GPU worker is actually a bit slower than a CPU core. AI wins by width, not speed: the math is thousands of independent sums, and a GPU runs them all at once instead of in rounds.
• Fast workers are only half the story. Every cell's weight row has to arrive from memory first, and with thousands of workers firing at once, delivering weights becomes the slow part. How fast memory can feed the chip is called memory bandwidth.
• That's why AI chips are wrapped in stacks of special high-speed memory called HBM, and why in 2026 that memory, not just the GPU itself, is one of the scarcest parts in AI.
• An AI company says it is “GPU-constrained” and can't get enough chips, and a single high-end GPU costs far more than a faster CPU. Why pay for the GPU anyway?
• The core math is thousands of independent little sums that don't wait on each other, so a GPU's many workers do them all at once while a fast CPU plods through in rounds. Wide beats fast here, so demand for GPUs, not CPUs, is what's scarce, and in 2026 companies are as squeezed by the memory that feeds those chips and the power that runs them as by the chips themselves.

Key takeaway

AI math is parallel, so wide hardware wins.

What you can do after this lesson

You can explain why AI runs on GPUs: the matmul is parallel, so wide hardware wins.

Prefer reading first? This lesson has a companion explainer: GPUs for AI

Where it leads: But one chip can't hold a frontier model…

This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.