The edge of the map: what LLMs can't do

What this lesson shows

Every LLM limit traces back to one fact: it predicts plausible tokens, not truth.

The question it opens with

You now know how the whole machine works. So where does it hit a wall, and why, straight from the mechanism?

The walkthrough, in the lesson's own words

• Now swap in bigger or smaller factors, the drift grows with the numbers.
• The model just answered this multiplication. Tap its answer to check it.
• Tell it your name, then come back tomorrow. Does it remember?
• Tap each wall to see the mechanism behind it, and the fix.
• You can now predict failure before it happens.
• It didn't compute the product, it pattern-matched digits that look about right, and stated the miss with total confidence.
• It pattern-matches digits that look about right instead of running the multiplication, so it lands close but wrong, and says it with total confidence. A fact it has seen a thousand times it can repeat; an exact product it has to compute, which it can't, so it fakes it.
• New chat tomorrow: “What's my name?” What does it say?
• Right, nothing persists between chats; a new one starts blank.
• Almost, within one chat it remembers; across chats it doesn't.
• Your name only “stuck” while it sat inside the context window (5.3); once that scrolls away, the model is blank again. The illusion of memory is just the transcript being re-fed each turn, so within one chat it can repeat your name, but a fresh conversation starts with nothing.
• If your ChatGPT does remember your name in a new chat, that is a memory feature saving notes outside the model and re-feeding them each time; the model itself still starts blank.
• None of these are mysteries anymore, each has a fix.
• Each limit is the flip side of how the machine works, and each has a fix: tools, retrieval, verification, or a human. That's the payoff of understanding the mechanism, you stop being surprised by failures and start designing around them.
• A teammate wants to ship a feature where the model totals up an invoice and states the figures as fact, with no checking. Which limits make this risky, and what would you do instead?
• This hits two walls: shaky exact arithmetic (it pattern-matches digits instead of computing) and no guaranteed truth (a fluent number can be confidently wrong). Route the math to a real calculator or tool, and verify any load-bearing figure before it counts. The point of knowing the mechanism is to predict the failure and design around it.

Key takeaway

You can predict where an LLM will fail before it does, and reach for the right fix (tools, retrieval, verification, a human).

What you can do after this lesson

You can name the fundamental limits of LLMs and tie each one to the mechanism that causes it.

Prefer reading first? This lesson has a companion explainer: LLM limits

Where it leads: One gate left: prove you can make the ship call. Then, the hardware underneath it all.

This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.