If it only predicts one token, how does it write a whole essay?

It runs the same step over and over. Each token it produces is appended to the text and fed back in, so the next prediction is conditioned on everything including what it just wrote. Structure appears because coherent continuations are the likely ones.

Does the model plan a sentence before writing it?

There is no separate planning stage in the architecture. Interpretability work does find internal state that anticipates several tokens ahead, and reasoning models get an explicit scratchpad to think in, but the output loop is still one token at a time.

Why does the same prompt give different answers?

Because the final step samples from a distribution instead of always taking the top token. Set temperature to zero and most models become close to repeatable, though providers do not guarantee it: batching and hardware can change the arithmetic slightly.

Plain-language explainer

Next-token prediction, explained

Is ChatGPT really just predicting the next word?

Yes, and that is not a small thing. The entire model is trained on one objective: given text, guess the next token. Do that near-perfectly across trillions of tokens and the guessing has to absorb grammar, facts, style, code idioms and chains of reasoning-shaped text, because they all improve the guess. Chat is the same trick pointed at a transcript: the model predicts what a helpful assistant would say next, one token at a time.

Last reviewed July 25, 2026

Do not just read it. Operate the mechanism yourself in a short interactive lesson.

See it work: Predict the next word →

Free, no code, no signup.

What people get wrong

It plans the whole answer before writing. It commits one token at a time; each choice constrains the next.
Prediction means memorization. The model mostly generalizes patterns; reproducing long passages verbatim is the exception, not the mechanism.
'Just autocomplete' means it cannot be capable. The ceiling of the trick rises with scale, which is exactly what the last few years demonstrated.

Where you see it in real products

The word-by-word streaming you watch in chat apps is the loop, live.
Your phone keyboard's suggestions are a tiny cousin of the same idea.
APIs price output per token because tokens are what the model produces, one at a time.

Common questions

If it only predicts one token, how does it write a whole essay?: It runs the same step over and over. Each token it produces is appended to the text and fed back in, so the next prediction is conditioned on everything including what it just wrote. Structure appears because coherent continuations are the likely ones.
Does the model plan a sentence before writing it?: There is no separate planning stage in the architecture. Interpretability work does find internal state that anticipates several tokens ahead, and reasoning models get an explicit scratchpad to think in, but the output loop is still one token at a time.
Why does the same prompt give different answers?: Because the final step samples from a distribution instead of always taking the top token. Set temperature to zero and most models become close to repeatable, though providers do not guarantee it: batching and hardware can change the arithmetic slightly.

Related explainers

How LLMs work Transformers Temperature Softmax Reasoning models Speculative decoding

One idea at a time, in your inbox

New lessons and explainers, written the way these pages are. Now and then, not daily, and never a sales sequence.

We email you a confirmation link first. Unsubscribe in one click, any time. Privacy.

Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.