Why does output cost more than input?

Input is processed in one parallel pass, which uses the hardware efficiently. Output is produced one token at a time, and each token needs the model's weights read from memory again. The same accelerator produces far fewer output tokens per second than it can absorb input tokens, and the price follows.

What actually makes one request more expensive than another?

Prompt length, output length, and whether the model was asked to think first. A reasoning model can spend thousands of hidden tokens before it writes a short answer, so a two-line reply is not a guide to what it cost.

How do I cut my AI bill without hurting quality?

In order of payoff: route easy requests to a cheaper model, cache the stable prefix of your prompts, trim the context you send, and cap output length. Most large bills come from resending far more context than the task needs on every call.

Plain-language explainer

What one AI answer actually costs

Why is running AI so expensive, and what are you paying for?

Every answer is computed from scratch on hardware that has to be reserved whether or not you are using it. There is no cached page to serve. Cost scales with the tokens in your prompt plus the tokens generated, and generation is the expensive half because each new token requires reading the entire model out of memory again. That is why prices are quoted per token, why output costs several times more than input, and why long context and long thinking both show up on the bill.

Last reviewed July 25, 2026

Reading is the slow way. Start with a free lesson you can operate right now.

Start free: Why GPUs beat CPUs →

Free, no code, no signup.

Then go deeper: Inference economics: what an answer costs Locked

What people get wrong

Costs are dominated by training. Training is a huge one-off, and for a popular product the cumulative cost of serving answers overtakes it.
Prices only go down. Per-token prices have fallen steeply, and reasoning models spend far more tokens per answer, so the cost of a task has not fallen as fast as the cost of a token.
A free tier is free to run. It is a marketing cost paid in GPU time, which is why free tiers get smaller models, tighter limits and lower priority.

Where you see it in real products

Message caps and slowdowns on consumer subscription plans.
Per-token API pricing with output priced above input, and a discount for cached input.
Cheap and expensive model tiers offered side by side for the same task.

Common questions

Why does output cost more than input?: Input is processed in one parallel pass, which uses the hardware efficiently. Output is produced one token at a time, and each token needs the model's weights read from memory again. The same accelerator produces far fewer output tokens per second than it can absorb input tokens, and the price follows.
What actually makes one request more expensive than another?: Prompt length, output length, and whether the model was asked to think first. A reasoning model can spend thousands of hidden tokens before it writes a short answer, so a two-line reply is not a guide to what it cost.
How do I cut my AI bill without hurting quality?: In order of payoff: route easy requests to a cheaper model, cache the stable prefix of your prompts, trim the context you send, and cap output length. Most large bills come from resending far more context than the task needs on every call.

Related explainers

GPUs for AI KV cache Model routing Training vs inference

More in Speed, cost and control

Part of See How AI Works, a free interactive course, where you learn how modern AI works by operating it, not watching videos.