Plain-language explainer
The transformer, explained
What is a transformer, and how does the architecture actually work?
A transformer is a stack of identical blocks, each doing two things. First, attention: every token looks at the earlier tokens and pulls in what it needs, so meaning flows between words. Second, a feed-forward network transforms each token on its own, where most of the model's learned knowledge lives. A running representation of each token passes through the stack, refined a little per block, until the top of the stack is sharp enough to score every possible next token.
Do not just read it. Operate the mechanism yourself in a short interactive lesson.
See it work: The transformer block, assembled & stacked →Free, no code, no signup.
What people get wrong
- A transformer is one big tangled network. It is the same small two-part block, repeated dozens of times in a stack.
- Attention is the whole story. Attention routes information, but most parameters sit in the feed-forward layers that transform it.
- Understanding happens in one pass of reading. Each block re-mixes the tokens again, so the representation sharpens layer by layer.
Where you see it in real products
- The T in GPT stands for transformer.
- Nearly every modern AI model, chat, code, image and audio included, is built on this architecture.
- Model cards list layer counts and sizes: those numbers describe this exact stack.
Related explainers
Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.