Predict the next word, with no context

1. 1Strip away meaning. This model knows only word frequency. Tap the word you think it picks.
2. 2It picks the same word for every blank, because the most common word wins regardless of the sentence.
3. 3Let it read the two words before the blank and watch a rare word shoot to the top.

A language model does one thing: it ranks every word it knows by how likely that word is to come next, and picks from the top of the list. This widget shows what that ranking looks like when the model is not allowed to read the sentence. The only information left is how often each word appears in English, so the word at the top is always the most common word in the language, and it wins every blank, every time.

Then you give the model the two words before the blank. Now the list is conditioned on context. A word that is rare in general, like drinks, is very likely after The cat, and it jumps from the bottom of the list to the top. Nothing about the model changed. The only difference is what it was allowed to read before ranking.

That is the entire idea behind predict-then-reveal: you commit to a guess about what the machine will do, and the machine answers you. Most people guess that the model picks a sensible word. It does not. Without context it picks the, and it keeps picking the.

Everything in a modern language model is a way of using more context better. Embeddings turn words into positions so context can be compared. Attention lets each word look at the others. Longer context windows let the model read more before it ranks. When someone says an LLM looked something up or knows a fact, this is the mechanism underneath: a ranked list of next words, computed from whatever text sits in front of it. That is also why it can sound sure and still be wrong.

Paste this into any HTML page, course platform, wiki or slide tool that accepts an iframe. It runs with no account, sets no cookies of its own, and links back here.

Attribution is built in. If you write about it, a link to this page is all we ask.

Why does the model keep picking the word the?

Because with no context the only signal left is raw frequency, and the is the most common word in English. Every blank gets the same top word until the model is allowed to read the words around the blank, at which point the ranking changes completely.

Is this how ChatGPT actually works?

At the core, yes. A large language model produces a probability for every token in its vocabulary and picks from that list, one token at a time. The difference is scale: it reads thousands of tokens of context through attention, and its ranking comes from billions of trained weights rather than a frequency table.

Can I embed this widget in my own page or slides?

Yes. Copy the iframe snippet on this page into any HTML page, a course site or a notebook. It needs no account, sends no personal data, and links back here so your readers can go deeper.