Model routing: the cheapest model that passes
Every request hits your biggest, priciest model. The bill and the latency are brutal, and most of those requests were easy.
The idea inside
Route each task to the cheapest model tier that still clears it; cascade when difficulty is unknown, and fall back when a tier fails.
After this lesson
You can route a workload across model tiers to hit a quality bar at the lowest cost and latency, using cascades and fallbacks.
Where it leads
Inside this lesson
That's the real lesson stage, paused. Claim your pass to operate it.
See how AI actually works, end to end.
This lesson is one stop on the full arc. Unlock all of it, and keep it for life.
What you get
- The 34-lesson main path, a finishable route from a word to agents
- Goal tracks for using AI at work and building AI features
- Boss labs that make you apply a whole act, not just recognize it
- Spaced recall that brings each idea back before you forget
- Course memory: every term defined, with links to where it first appears
- A shareable capability card when you finish the main path
- Lifetime access on every device, every future lesson included
Not videos to watch. You predict, operate the machine, then prove it. That is why it stays.
99 interactive lessons and challenges. No videos, no code.
Free launch pass: lifetime access, no card needed
New here? The first lessons are free to try. Start with lesson 0.1
What this lesson shows
Route each task to the cheapest model tier that still clears it; cascade when difficulty is unknown, and fall back when a tier fails.
The question it opens with
Every request hits your biggest, priciest model. The bill and the latency are brutal, and most of those requests were easy.
The walkthrough, in the lesson's own words
- Route each task to a tier. Hold quality at 100% and get under both budgets.
- You can't know each task's difficulty up front. Predict what a cascade costs.
- There is no single best model. You route by difficulty, under a budget.
- 100% quality, well under both budgets. Each task went to the cheapest tier that can still clear it. That is routing.
- A tier below a task's difficulty can't solve it, so quality drops. Move that task up a tier.
- Quality is perfect but you're over budget. The default sends everything to Frontier. Route the easy, high-volume tasks down.
- Send each task to the cheapest tier whose difficulty still covers it. The 6,000 easy tickets only need Small. The mid tasks need Mid. Only the 500 hard analyses need Frontier. Same quality, a fraction of the cost.
- Costs and latencies here are illustrative compute units, not real prices, but the trade-off is real.
- A cascade tries the Small model first and escalates to Frontier only when Small is unsure. Across 100 mixed requests, does the cascade cost MORE or LESS than sending all 100 to Frontier?
- Most requests are easy, so Small clears them and never reaches Frontier. You pay the frontier price only for the few hard ones that escalate.
- Same idea in reverse for reliability: a fallback. If Frontier times out or errors, drop down to Mid so the request still gets an answer instead of failing.
- The biggest, smartest model is always the right call.
- You route by difficulty under a budget: the cheapest tier that still passes, with a cascade for unknowns and a fallback for failures.
- A support assistant feels instant even at huge scale, and still nails the rare hard question. What's the most likely trick under the hood?
- Routing: ~90% of tickets go to a small, fast model, and only the hard ~10% escalate to a frontier model. Snappy products at scale are usually smart routing, not one giant model answering everything.
- You can route a workload across model tiers to hit a quality bar at the lowest cost and latency, instead of overpaying the frontier model for easy work.
- Cascade total ≈ 700 vs all-Frontier 2,000. Less, because most never escalate.
Key takeaway
You held quality at 100% while cutting cost and latency, by routing easy work to small models and saving the frontier model for the hard few.
What you can do after this lesson
You can route a workload across model tiers to hit a quality bar at the lowest cost and latency, using cascades and fallbacks.
Prefer reading first? This lesson has a companion explainer: Model routing
This is the written summary. The lesson itself is interactive: you predict, drag and operate the mechanism above, and the reveal answers you.