Plain-language explainer
Scaling laws, explained
Why did making models bigger keep making them better?
Researchers found that a model's error falls along smooth, predictable curves as you grow three things: parameters, training data and compute. Predictable is the key word: labs could forecast how good a run would be before spending on it, which justified spending more. The Chinchilla result added the balance: for a fixed compute budget, a smaller model trained on more data beats a bigger one trained on less, roughly twenty tokens of data per parameter.
Do not just read it. Operate the mechanism yourself in a short interactive lesson.
See it work: Scaling laws →Free, no code, no signup.
What people get wrong
- Scaling laws promise intelligence. They predict next-token error. Useful abilities ride along, but unevenly and less predictably.
- Bigger is always the answer. Chinchilla showed several famous models were undertrained: too many parameters for their data.
- Scaling is finished. The curves still hold, but the axes multiplied: data quality, post-training and inference-time compute now scale too.
Where you see it in real products
- Model families ship in size tiers because cost and quality trade along these curves.
- 'Trained on X trillion tokens' in model cards is the data axis of the law.
- Efficiency headlines, like strong models trained unusually cheaply, are wins against these same curves.
Related explainers
Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.