Plain-language explainer
RLHF, explained
What is RLHF, and why does a text predictor act like an assistant?
Pretraining produces a raw predictor that continues text in any direction, helpful or not. RLHF is the finishing school: humans compare pairs of model answers and pick the better one, a reward model learns to imitate those preferences, and the LLM is then trained to score highly on that reward. The result answers questions, follows instructions and declines harmful requests, not because it was told rules, but because responses shaped that way rated better.
Do not just read it. Operate the mechanism yourself in a short interactive lesson.
See it work: RLHF / post-training →Free, no code, no signup.
What people get wrong
- RLHF teaches the model facts. Knowledge comes from pretraining. RLHF shapes behavior, tone and refusals.
- The assistant has values it reasons from. It has a policy optimized to produce answers humans preferred.
- RLHF makes models more truthful. It can reward pleasing over accurate, which is where sycophancy comes from.
Where you see it in real products
- The thumbs up and down buttons collect exactly this preference data.
- 'Which response do you prefer?' side-by-side prompts in chat apps feed the reward model.
- The gap between a raw base model and the polite assistant you use is mostly this step.
Related explainers
Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.