What did the scaling laws actually predict?

That loss falls in a smooth, predictable curve as you increase model size, data and compute together. That predictability is what justified spending hundreds of millions on a single training run: you could forecast roughly what you would get before you started.

What did Chinchilla change?

It showed the models of the time were oversized for the amount of data they were trained on. For a fixed compute budget, a smaller model trained on more tokens did better. Headline model sizes stopped racing upward and training-token counts shot up instead.

Have scaling laws stopped working?

The compute-to-loss curve still holds. What changed is where the cheapest gains are. Labs now also buy quality through better data, heavier post-training, and spending compute at answer time rather than only during training.

Plain-language explainer

Scaling laws, explained

Why did making models bigger keep making them better?

Researchers found that a model's error falls along smooth, predictable curves as you grow three things: parameters, training data and compute. Predictable is the key word: labs could forecast how good a run would be before spending on it, which justified spending more. The Chinchilla result added the balance: for a fixed compute budget, a smaller model trained on more data beats a bigger one trained on less, roughly twenty tokens of data per parameter.

Last reviewed July 25, 2026

Reading is the slow way. Start with a free lesson you can operate right now.

Start free: From the internet to your answer →

Free, no code, no signup.

Then go deeper: Scaling laws Locked

What people get wrong

Scaling laws promise intelligence. They predict next-token error. Useful abilities ride along, but unevenly and less predictably.
Bigger is always the answer. Chinchilla showed several famous models were undertrained: too many parameters for their data.
Scaling is finished. The curves still hold, but the axes multiplied: data quality, post-training and inference-time compute now scale too.

Where you see it in real products

Model families ship in size tiers because cost and quality trade along these curves.
'Trained on X trillion tokens' in model cards is the data axis of the law.
Efficiency headlines, like strong models trained unusually cheaply, are wins against these same curves.

Common questions

What did the scaling laws actually predict?: That loss falls in a smooth, predictable curve as you increase model size, data and compute together. That predictability is what justified spending hundreds of millions on a single training run: you could forecast roughly what you would get before you started.
What did Chinchilla change?: It showed the models of the time were oversized for the amount of data they were trained on. For a fixed compute budget, a smaller model trained on more tokens did better. Headline model sizes stopped racing upward and training-token counts shot up instead.
Have scaling laws stopped working?: The compute-to-loss curve still holds. What changed is where the cheapest gains are. Labs now also buy quality through better data, heavier post-training, and spending compute at answer time rather than only during training.

Related explainers

Gradient descent Transformers Mixture of Experts Distillation Reasoning models

One idea at a time, in your inbox

New lessons and explainers, written the way these pages are. Now and then, not daily, and never a sales sequence.

We email you a confirmation link first. Unsubscribe in one click, any time. Privacy.

Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.