Plain-language explainer
Gradient descent, explained
How does a neural network actually learn?
Training is a loop. The model makes a prediction, a loss function measures how wrong it was, and calculus tells every single weight which tiny nudge would have made the error smaller. All the weights take their nudge, and the loop repeats, billions of times. That is gradient descent: rolling downhill on an error landscape you can never see whole, one small step at a time. Nobody writes the rules. The rules are whatever weight values the descent settles into.
Do not just read it. Operate the mechanism yourself in a short interactive lesson.
See it work: Gradient descent: rolling downhill →Free, no code, no signup.
What people get wrong
- Engineers program the model's knowledge. They program the loop. The knowledge is the byproduct of running it on data.
- Training finds the perfect answer. It finds a low valley that works, not a guaranteed best one.
- The model keeps learning when you chat with it. After training, the weights are frozen. Your conversation changes the context, never the weights.
Where you see it in real products
- Headlines about training runs costing millions describe this loop at scale.
- The loss curves in AI lab posts are the downhill path, drawn over time.
- Fine-tuning services run the same loop, starting from the finished model and using your data.
Related explainers
Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.