📉The metric almost everyone uses is the wrong one
Cost per API call is easy to compute and tells you almost nothing. An agent that costs a fraction of a cent per call but silently retries four times, escalates a third of its cases to a human, and produces answers users reject is not cheap. It is expensive in a way the invoice does not show you.
The number that decides whether an agent is worth running is cost per successfully completed task, measured against the cost of the manual process it replaces. Everything below is a way of getting to that number before you commit engineering time.
If your AI budget line is "tokens × rate", you are pricing the happy path. Real agent workloads spend most of their money on the cases that did not go smoothly.
💵Start with real published rates
Published list pricing for the Claude model family, per million tokens, as of July 2026. We use Claude here because it is the family we deploy most; the framework is provider-agnostic and the arithmetic works the same against any published rate card. Always check the provider's current pricing page before budgeting — these rates move.
| Model | Input / 1M | Output / 1M | Typical role in an agent |
|---|---|---|---|
| Claude Haiku 4.5 | $1.00 | $5.00 | Classification, routing, extraction |
| Claude Sonnet 5 | $3.00 | $15.00 | The workhorse — most agent turns |
| Claude Opus 5 | $5.00 | $25.00 | Hard reasoning, planning, review |
| Claude Fable 5 | $10.00 | $50.00 | The genuinely hard minority of tasks |
🔁The retry multiplier — where budgets actually break
This is the part almost every estimate omits, and it is the single biggest driver of real-world agent cost.
If an agent succeeds on a given task 70% of the time and simply retries on failure, the naive assumption is that you pay 1 ÷ 0.7 ≈ 1.43× the single-attempt cost. That is wrong in both directions, and understanding why is most of the work.
It understates cost because a failed attempt is rarely cheap. A failure usually happens late — after the agent has read the context, made several tool calls and produced most of an answer. You pay nearly the full task cost for the failure, then pay it again on the retry. And retries are not independent events: the cases that fail are the hard ones, so a second attempt on a failed task succeeds at a lower rate than the headline 70%.
It can also overstate cost, because a well-built agent does not blindly retry. It escalates. An agent that recognises it is stuck and hands off after one attempt costs less in inference than one that grinds through four — but moves cost to a human instead. That is often the right trade; it just needs to appear in the model rather than being invisible.
Model the failure cost separately
A failed attempt typically costs 60–90% of a successful one, because failure happens late. Assume full cost unless you have data showing otherwise — it is the conservative direction.
Degrade the success rate on retry
The population of failed tasks is harder than the population as a whole. If overall success is 70%, second-attempt success on the failures is materially lower. Do not reuse the headline rate.
Price escalations, do not hide them
An escalation is a real cost — a human minute at a loaded rate. An agent with a 90% resolution rate and a clean escalation path is often cheaper all-in than one at 97% that burns tokens grinding on the last few percent.
Cap it, and alert on the cap
Hard limits on iterations, tool calls and spend per run, enforced server-side. An agent that stays under its per-run cap but hits it constantly is failing — the cap should raise an alert, not just silently truncate.
🔀Routing: the highest-leverage cost decision
Look at the pricing table again. The spread between the cheapest and most capable model in the family is 10× on input and 10× on output. Most agent turns do not need the top of that range.
A typical support-resolution agent turn decomposes into several distinct steps: classify the intent, decide which tool to call, read the tool result, decide whether the answer is sufficient, and write the reply. Only one or two of those genuinely require frontier reasoning. Routing classification and tool-selection steps to a small model and reserving the expensive model for the hard decision is the single largest structural saving available, and it is an architecture decision rather than a prompt tweak.
The constraint is that routing has to be measured, not assumed. A cheap model that misclassifies drives retries, and retries cost more than the model you saved on. This is precisely why cost per successful outcome is the metric — a routing change that lowers cost per call while raising the retry rate is a regression that a cost-per-call dashboard will report as an improvement.
⚡Caching economics: the part with hard numbers
Prompt caching has published, unambiguous multipliers, which makes it the one part of agent cost you can compute exactly rather than model.
A cache read costs roughly 0.1× the base input price. A cache write costs 1.25× with a five-minute time-to-live, or 2× with a one-hour TTL. That gives a clean break-even calculation that most teams never do.
Caching is a prefix match — order decides everything
Any byte change anywhere in the prefix invalidates everything after it. A timestamp interpolated into the top of a system prompt makes the entire rest of that prompt uncacheable, no matter how the caching is configured.
The silent invalidators are always the same handful
A current date in the system prompt, a request ID early in the content, non-deterministic JSON serialisation, or a tool list that varies per user. Each one quietly costs full price on every request.
Verify rather than assume
The response reports cached versus uncached token counts directly. If the cached figure is zero across repeated requests with what you believe is an identical prefix, something in that prefix is changing. Check before optimising anything else.
| TTL | Write cost | Read cost | Break-even | Use when |
|---|---|---|---|---|
| 5 minutes | 1.25× | ~0.1× | 2 requests (1.25 + 0.1 = 1.35× vs 2× uncached) | Continuous traffic — requests arrive closer together than the TTL |
| 1 hour | 2× | ~0.1× | 3 requests (2 + 0.2 = 2.2× vs 3× uncached) | Bursty traffic with long idle gaps between bursts |
In our experience the caching audit is the cheapest win available on an existing agent: no architecture change, no quality risk, and it is usually a prompt-assembly reordering rather than new code.
🧮A worked model — with the assumptions stated
Below is the shape of the estimate we build during scoping. Read it as a template for your own numbers, not as a measurement of ours — the token counts and rates are illustrative assumptions, and we say so plainly because a benchmark you cannot reproduce is worth nothing.
Take a support-resolution agent. Per attempt, assume it reads a system prompt and policy context, pulls the customer record and order history through two or three tool calls, reasons over the result, and writes a reply. Now build the estimate in this order:
1. Tokens per successful attempt
Split input and output, and split cached input from uncached. The system prompt and policy text should be cached; the customer record should not be. Measure this with a token-counting call against real traffic rather than estimating from character counts.
2. Cost per attempt at your routing mix
Apply the per-model rates to the steps each model actually handles. If 70% of steps route to a small model, the blended rate is far below the headline rate of your most capable model.
3. Multiply by the real attempt count
Not 1. Use your measured success rate, with a degraded rate for retries on already-failed tasks, plus the cost of any attempt that ends in escalation.
4. Add the human cost of escalations
Escalation rate × average human handling minutes × loaded hourly rate. This line is frequently larger than the inference line, and omitting it is what makes agent business cases look better than they are.
5. Compare against the manual baseline
The same task done entirely by a person, at the same loaded rate. This is the only comparison that answers the actual question, and it is the one most vendor cost models skip.
If steps 3 and 4 are missing from a cost model you have been given, the model is describing the happy path. Ask for them before approving the budget.
📊What to instrument from day one
None of the above is worth much without measurement, and measurement has to be designed in rather than retrofitted. These are the metrics we wire into every agent we ship, because each one changes a decision.
Cost per successful outcome
The headline number. Everything else is diagnostic detail supporting this one.
Success, retry and escalation rates
The three inputs to the multiplier. Track them per intent or task type, not just in aggregate — the average hides the expensive minority.
Cached vs uncached input tokens
A cache hit rate that quietly drops to zero after a deploy is a common and entirely invisible cost regression.
Spend per run, with a hard ceiling
Enforced outside the model. Breaching it should terminate the run and raise an alert rather than degrading silently.
Cost distribution, not just the mean
Agent cost is long-tailed. The mean is reassuring and the 95th percentile is what actually appears on the invoice.