Prompt caching: reuse the prefix

What this lesson shows

Cache the stable prefix: the model processes it once, then reuses it far more cheaply.

The question it opens with

You re-send the same long system prompt and files every single turn. Isn't that wasteful?

The walkthrough, in the lesson's own words

• This is why long chats answer fast and giant system prompts are affordable. Drag the turns, then toggle caching.
• Predict: after how many reuses does paying to cache the prefix pay off?
• Every turn re-sends the same long system prompt, tools, and files. Wasteful?
• Stable content first, byte-for-byte. Then reuse it for almost nothing.
• Every turn re-sends the identical prefix, and without caching you pay to re-read all of it, again and again. Both bars sum that input cost; drag the turns and watch the gap widen.
• A cache hit needs a byte-for-byte match. Editing the start of the prompt changes it, so the cache no longer matches and the prefix is re-written at 1.25×. From there you pay full price until the cache re-warms, which is why you keep stable content first and never touch it.
• Caching isn't free: the first turn pays 1.25× to write the prefix. But every later turn reads it at 0.1×, a 90% discount.
• So after how many reuses does the cache start saving you money?
• Just one. The extra 0.25× write is tiny; a single 0.1× read instead of a full 1× re-bill already wins.
• The big blue prefix is identical every turn; only the thin message changes. That identical bulk is exactly what gets cached and reused.
• The prefix is the system prompt, tool definitions, and attached files, the stable scaffolding from the context window (lesson 6.2). It can be thousands of tokens. The latest user message is usually tiny by comparison, yet a fresh request re-bills the whole thing from scratch each turn.
• Two things the cache is not: it isn't memory about you, it stores the model's processed reading of the prefix (the KV states from 4.2); and it isn't permanent, it expires in minutes, so a chat resumed after lunch pays the write again.
• Cached, the prefix pays for itself after a single turn, as long as the start stays byte-for-byte identical. It's the same trick as
• You notice the agent re-reads the same big instructions every turn, yet turns stay fast and your usage limit is not shredded. Why?
• Prompt caching: the shared prefix is billed and processed once, then reused at about 90% off as long as it stays byte-for-byte identical. That is also why teams building on these models structure prompts stable-part-first, so the cache keeps matching.

Key takeaway

Put stable content first and reuse it; caching turns repeated context from costly to nearly free.

What you can do after this lesson

You can explain prompt caching and why putting stable content first cuts cost and latency.

Prefer reading first? This lesson has a companion explainer: Prompt caching

Where it leads: Caching is a harness trick, what about the tools the model itself reaches for?

This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.