Token awareness & the economics

What this lesson shows

Every token in and out costs money and time, so fewer, well-chosen tokens beat more.

The question it opens with

Two prompts get the same answer, but one costs 10× and runs slower. Why?

The walkthrough, in the lesson's own words

• Drag the output length. Watch the dollar figure and the wait move.
• The big bars are a fixed all-off vs all-on comparison. Flip the levers to move your own per-turn bill below.
• Every token in and out is billed. Spend context on purpose.
• Representative mid-tier 2026 prices, real rates vary by model and provider.
• Output is billed at 5× the input rate, and it's what sets latency.
• History rides along every turn, so input creeps up as you go.
• Billing counts tokens, not answers. Output is billed at 5× the input rate and it's what you wait for, while input creeps up as the whole repo and chat history (6.2) ride along every turn.
• Both halves are billed. Output costs more per token, the model generates each one, and it's also what you wait for.
• Prompt caching (6.4): the stable prefix is read at 0.1×, 90% off the bulk you re-send.
• Let the agent fetch only the relevant chunks (6.9) instead of the whole repo, far fewer input tokens.
• Trim the context (6.3): keep the last couple of turns, not the whole chat.
• More context can help, but it also costs more, runs slower, and risks context rot (6.3), where the signal drowns in noise. So: cache the stable prefix, retrieve instead of dumping, prune stale history.
• When nobody is waiting on the answer, work can be queued for cheap overnight processing, like a million rows classified while you sleep. Batch APIs run those requests together at a large discount, trading immediacy for lower cost. Rule of thumb: pay the real-time premium only where a human is waiting, and push everything else to batch.
• Your agent session hits its usage limit mid-afternoon, and long answers feel slow. What is actually being counted?
• Tokens, in and out. Output is billed at several times the input rate, so long replies and re-read context burn through the quota, and the output is also what you wait for. The same arithmetic drives the API bill for teams that build on these models.
• More context can recall facts and tools the model would otherwise miss.
• Every extra token costs money and adds latency.
• Bloated context risks context rot (6.3), the answer drowns in irrelevant tokens.

Key takeaway

You can treat cost, latency, and quality as one budget and spend tokens where they pay off.

What you can do after this lesson

You can reason about token cost and latency, and make deliberate trade-offs (caching, retrieval, pruning).

Where it leads: You can drive the tools, but what actually happens on each turn of the loop?

This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.