Why do models invent citations and sources?

Because a plausible-looking citation is exactly the pattern the surrounding text calls for, and nothing in the loop checks that it exists. The shape of a reference is easy to predict, the specific true one is not, and the model has no way to tell the two apart.

Does retrieval (RAG) fix hallucination?

It reduces it substantially and does not eliminate it. Grounding an answer in fetched text removes the need to invent, but the model can still misread a passage or fill a gap when retrieval comes back empty. Citations you can click are the practical defence.

Can a model tell me when it is unsure?

Only roughly. Models are trained to express uncertainty in words, and that expression is itself predicted text rather than a measured confidence. Treat hedging as a weak signal and verify anything that matters.

Do reasoning models hallucinate less?

Not automatically. Thinking longer helps on problems where reasoning can catch an inconsistency, but several reasoning-focused models have been reported to fabricate more on simple factual-recall tests than their non-reasoning siblings. More thinking is not more knowing. Grounding in sources is what moves factual reliability.

Which tasks are riskiest for hallucination?

Open-ended factual questions about the long tail: obscure people, small companies, niche research, fresh events, precise numbers. The risk is lowest when the model transforms text you supplied, a summary, a rewrite, a translation, because the truth is sitting in the prompt.

Is 'hallucination' even the right word?

Many researchers dislike it, since the model does not perceive anything, and 'confabulation', inventing a fluent story to fill a gap, fits better. The term stuck because it is vivid. Whatever the label, the mechanism is ordinary prediction doing its job past the edge of its knowledge.

How do companies measure hallucination?

With evals: fixed question sets where the true answer is known, graded automatically or by humans, plus faithfulness checks that compare a summary against its source document. Public leaderboards do this across models, and serious teams run the same kind of eval on their own product traffic.

Plain-language explainer

Why AI hallucinates

Why do AI models confidently make things up?

Because they are built to continue text plausibly, not to report facts. A language model always produces the most likely next words. When the truth is in its training data, likely and true usually coincide. When it is not, the model fills the gap with something that sounds right, in perfect fluent prose, because nothing inside it distinguishes a remembered fact from a plausible pattern. Hallucination is not a glitch on top of the system. It is the system, running without grounding.

Last reviewed August 4, 2026

Reading is the slow way. Start with a free lesson you can operate right now.

Start free: Predict the next word →

Free, no code, no signup.

Then go deeper: Hallucination Locked

What counts as an AI hallucination?

A hallucination is output that is presented as fact but is false or unsupported. The word covers two different failures, and telling them apart matters. The first is fabrication: the model asserts something about the world that is simply not true, a wrong date, an invented statistic, a book that does not exist. The second is a faithfulness failure: you gave the model a document or a search result, and its summary says something the source does not. The first is a gap in what the model absorbed from training. The second is a misreading of text sitting right in front of it.

The most recognizable subspecies is the invented citation: a case name, paper title or URL with perfect formatting and no existence. Citations get fabricated so readily because their shape is extremely predictable, author, year, plausible title, while the specific true reference is not. The model can always produce the shape. It cannot always produce the fact.

One more shape is worth naming: sycophantic error. Push back on a correct answer, and the model may fold and agree with your wrong version, because agreeable continuations were rewarded during training. The mistake reads as politeness, but it is the same underlying issue: the text that scores as most likely is not always the text that is true.

Why does it happen? The mechanics

A language model answers by predicting the next token, one after another, given everything so far. There is no database lookup, no list of verified facts, and no step where the sentence is compared with reality. Facts the model saw many times in training are encoded strongly and come out reliably. Facts it saw once, or never, sit in the same statistical machinery as everything else, and when the prompt demands an answer, the machinery produces the most plausible continuation it can. Plausible and true overlap most of the time. Hallucination lives in the gap.

Training incentives make it worse. A 2025 OpenAI research paper argued that models hallucinate partly because of how they are trained and evaluated: most benchmarks grade answers right or wrong, with no credit for saying 'I don't know'. Under that scoring, a model that guesses beats a model that abstains, the same way a student who never leaves a multiple-choice question blank outscores one who does. Models are, in effect, optimized to be confident test-takers.

This is also why hallucinations come out fluent. The model's whole skill is producing likely text, and assured, well-structured prose is the likely shape of an answer. Confidence in the wording is a property of the writing, not evidence about the world. The interactive lesson on this site lets you watch a model keep generating smoothly right past the edge of what it knows, which is the whole phenomenon in one picture.

Real cases: when hallucination met the real world

In 2023, two New York lawyers filed a legal brief written with ChatGPT's help. It cited six court cases that did not exist, complete with convincing quotes and docket numbers. Opposing counsel could not find the cases because there was nothing to find, and a federal judge fined the lawyers. Courts in several countries have since sanctioned or reprimanded lawyers over fabricated AI citations, enough that many now require disclosure of AI assistance.

In 2024, a Canadian tribunal ordered Air Canada to honor a bereavement discount its website chatbot had invented. The airline argued the bot was responsible for its own words. The tribunal disagreed: the company published the answer, the company owns the answer. The case became the standard warning for anyone deploying a customer-facing bot without grounding it in real policy documents.

The pattern in both stories is the same and is the practical lesson: the text was fluent enough to pass a professional's glance. Hallucinations rarely look like errors. They look like everything else the model writes, which is why the defense has to be process, checking sources, grounding answers, rather than vibes.

How often do models hallucinate?

There is no single rate, because the rate depends on the task. Asked to summarize a document sitting in the prompt, current frontier models are wrong rarely: public faithfulness leaderboards have put the best models' summary-hallucination rates in the low single digits of percent. Asked open-ended questions about obscure people, niche products or fresh events, error rates climb steeply, because the long tail of the world is exactly what training data covers thinnest.

The trend line is real but partial. Each model generation has reduced hallucination on measured benchmarks, and grounded modes with web search and citations reduce it further. Two honest caveats survive the progress. First, reductions are uneven: some reasoning-focused models have been reported to hallucinate more than their predecessors on certain factual-recall benchmarks, even while getting better at math and code. Second, a lower rate on a benchmark is not a guarantee about your question. The risk concentrates precisely where you are least able to check.

What actually reduces hallucinations?

Nothing eliminates them, but a few practices cut the risk dramatically. The common thread is grounding: connect the model's answer to text that actually exists, and give it permission to not know.

Retrieval (RAG) helps because it changes the task from remembering to reading: the model answers from fetched text and can cite it. Failures then shift to retrieval quality, which you can measure and fix.
Lowering temperature does not fix hallucination. It makes output more repeatable, but the most likely continuation can be confidently wrong at any temperature.
Asking the same question twice and comparing answers is a cheap consistency check: fabrications tend to vary between runs, memorized facts tend to stay put.

Practical defenses, by situation. Process beats vibes.

Situation	What helps
Facts, dates, numbers	Use a grounded mode (web search or RAG) and ask for sources you can click. Then click one.
Citations and references	Never trust an unlinked citation. Ask for links, verify they resolve, and check the quote appears in the source.
Summaries of your documents	Paste or attach the document so the model reads it instead of remembering it, then spot-check claims against the original.
Math and counting	Have the model use a tool (calculator, code) rather than predict digits as text.
High-stakes calls (legal, medical, financial)	Treat the model as a drafting assistant, not an authority. A human verifies before anything ships.
Anything the model might not know	Say 'if you are not sure, say so' in the prompt. It is imperfect, but it measurably invites abstention.

Can hallucination ever be fully fixed?

Not in the current paradigm, and the honest framing is management rather than cure. A next-token predictor has no internal line between pattern and fact, so some rate of confident error is structural. What can change, and has been changing, is everything around the predictor: training that rewards calibrated uncertainty instead of blind guessing, evaluations that stop penalizing 'I don't know', products that ground answers in retrieved sources by default, and tool use that replaces prediction with lookup for the things lookup does better.

For a user, the practical takeaway is a calibration habit, not fear. Treat a model like a brilliant, fast, occasionally wrong colleague: wonderful for drafts, explanations, brainstorming and transformation of text you provide, and never the final authority on a fact you have not checked. The moment an answer matters, the question to ask is not 'does this sound right?' but 'what is this grounded in?'

What people get wrong

Hallucination is a bug that will be patched out. It is inherent to next-word prediction. It can be reduced with grounding and citations, not deleted.
The model knows when it is making things up. There is no internal fact-checker. Fluency and confidence are properties of the text, not evidence about truth.
Wrong answers mean bad training data. Even perfect data cannot cover everything, and the model fills every gap by design.
Lowering the temperature stops hallucinations. It only makes output more repeatable. A confident error at temperature 0 is still a confident error.

Where you see it in real products

Lawyers have been sanctioned for filing briefs with invented case citations from a chatbot.
An airline's support bot invented a refund policy, and a tribunal held the company to it.
The 'check important info' notice under chat boxes exists precisely because of this.
Search-grounded modes with clickable citations are the product answer: they turn 'trust me' into 'here is the source'.

Common questions

Why do models invent citations and sources?: Because a plausible-looking citation is exactly the pattern the surrounding text calls for, and nothing in the loop checks that it exists. The shape of a reference is easy to predict, the specific true one is not, and the model has no way to tell the two apart.
Does retrieval (RAG) fix hallucination?: It reduces it substantially and does not eliminate it. Grounding an answer in fetched text removes the need to invent, but the model can still misread a passage or fill a gap when retrieval comes back empty. Citations you can click are the practical defence.
Can a model tell me when it is unsure?: Only roughly. Models are trained to express uncertainty in words, and that expression is itself predicted text rather than a measured confidence. Treat hedging as a weak signal and verify anything that matters.
Do reasoning models hallucinate less?: Not automatically. Thinking longer helps on problems where reasoning can catch an inconsistency, but several reasoning-focused models have been reported to fabricate more on simple factual-recall tests than their non-reasoning siblings. More thinking is not more knowing. Grounding in sources is what moves factual reliability.
Which tasks are riskiest for hallucination?: Open-ended factual questions about the long tail: obscure people, small companies, niche research, fresh events, precise numbers. The risk is lowest when the model transforms text you supplied, a summary, a rewrite, a translation, because the truth is sitting in the prompt.
Is 'hallucination' even the right word?: Many researchers dislike it, since the model does not perceive anything, and 'confabulation', inventing a fluent story to fill a gap, fits better. The term stuck because it is vivid. Whatever the label, the mechanism is ordinary prediction doing its job past the edge of its knowledge.
How do companies measure hallucination?: With evals: fixed question sets where the true answer is known, graded automatically or by humans, plus faithfulness checks that compare a summary against its source document. Public leaderboards do this across models, and serious teams run the same kind of eval on their own product traffic.

Related explainers

RAG Temperature How LLMs work Context window

One idea at a time, in your inbox

New lessons and explainers, written the way these pages are. Now and then, not daily, and never a sales sequence.

We email you a confirmation link first. Unsubscribe in one click, any time. Privacy.

Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.