From predictor to assistant

What this lesson shows

After pre-training comes post-training: fine-tune on answers, then rank by human preference.

The question it opens with

Trained only to predict the next word, a raw model answers “Write a poem about cats” by listing more prompts, not a poem. So why does a real chatbot actually help?

The walkthrough, in the lesson's own words

• Tap the line you think the raw model writes next.
• Same prompt, two models. Pre-trained just continues; post-trained answers.
• You're the human rater. Pick the better reply, twice.
• Same weights, new behavior. That's post-training.
• Right, your line is just text to autocomplete, and online, instructions are followed by MORE instructions. So it continues the list, not obeys it.
• Not quite, the raw model only autocompletes text, with no idea it should obey. Online, instructions are followed by MORE instructions, so it continues the list.
• A raw, pre-trained model (Act 2) has only ever learned one trick: predict the next word of whatever text it sees. It has no concept of a “user” to obey, your prompt is simply text to continue. And in its training data, a line like an instruction is most often followed by yet another instruction, so the most likely continuation is more prompt-style text.
• Same weights, same architecture. Post-training adds almost no new knowledge; it mostly reshapes behavior.
• It's the SAME model, same weights, same architecture. Post-training added almost no new knowledge; it taught the predictor to prefer answering over autocompleting. Step one of that is SFT: fine-tune on example (prompt → good answer) pairs written by humans.
• After SFT, humans pick between two replies, that preference is the next signal.
• A handful of human picks like yours train a reward model, a judge that scores replies the way people would. Its scores then nudge the predictor's weights toward replies you'd prefer. That loop is RLHF.
• RLHF is now one member of a family. Labs often skip the separate judge and learn directly from preference pairs (DPO), use AI raters instead of people (RLAIF), or reward answers a program can check, like math and code (RLVR), which is how reasoning models learn to think. Same idea every time: prefer better replies.
• Not a new brain, the same next-word predictor, post-trained into an assistant.
• The assistant isn't a new brain, it's the same next-word predictor, post-trained. SFT shows it example answers; RLHF tunes it toward replies people prefer. That's where instruction-following, helpfulness, and refusals come from, barely any new facts, mostly new behavior.
• Turning a raw model into an assistant makes it smarter or teaches it new facts.
• Post-training mostly reshapes behavior and adds almost no new knowledge. It's the same predictor with the same weights, tuned to answer, follow instructions, and refuse, instead of blindly autocompleting.
• A chatbot follows your instructions, sounds polite, and politely declines harmful requests. None of that is a fact it looked up. Where does that behavior come from?
• From post-training on top of the raw predictor: SFT on example answers, then preference tuning (RLHF) toward replies people prefer. Same weights, new behavior, which is why two assistants on the same base model can act very differently.

Key takeaway

The assistant isn't a new brain; it's the same next-word predictor, post-trained to prefer the kind of answer you want.

What you can do after this lesson

You can explain post-training (SFT + preference tuning) and why a raw next-word predictor becomes a helpful, instruction-following assistant.

Where it leads: You know the whole recipe now. Prove you can run the training loop yourself, then we open up how it predicts across the whole sentence.

This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.