Why does AI give a different answer every time?
Because the last step is a dice roll, and there is a dial that decides how loaded the dice are. It is called temperature. Turn it below and watch the same prompt change its mind.
The cat drinks ___
- milk81%
- water15%
- cream4%
Middle: the likely word usually wins, but not always. This is the usual default.
What you just did
A language model does not choose a word. It produces a probability for every word it knows, and then something samples from that list. Temperature reshapes the list before the draw: low temperature exaggerates the gaps so the favourite dominates, high temperature flattens them so the long shots get a chance. It never changes which word is most likely, only by how much.
That is the whole reason the same prompt can give you a different answer twice, and the reason a model set to be creative is also a model set to be wrong more often.
Honest note: the maths here is real, p raised to 1/T then renormalised, and the counts come from actually sampling it. The starting percentages are hand-picked to be readable, not read off a specific model.
This is one dial out of a whole machine. The rest is a free course.
See where the probabilities come from →Free, no code, no signup.
Read the explainer: What temperature actually does · How next-word prediction works
One idea at a time, in your inbox
New lessons and explainers, written the way these pages are. Now and then, not daily, and never a sales sequence.
We email you a confirmation link first. Unsubscribe in one click, any time. Privacy.