The Cheapest LLM Can Cost More per Successful Task

A cheaper call can leave a more expensive task

Your team switches the first model in a support workflow to one with a lower token rate. The cost dashboard confirms that an individual call is cheaper. A month later, the bill has barely moved. The agent is retrying malformed answers, sending the same retrieved context again, and asking the old model to finish the cases that the new one cannot. Each call looks inexpensive in isolation. A customer ticket still needs one accepted answer, and the path to that answer now has more calls.

That is the question a price list cannot answer: how much did the entire workload spend for each task that passed its real acceptance check? The answer might favor the cheaper model. It might favor the more reliable route. You need both the cost of every attempt and the count of accepted outcomes to tell.

cost per successful task = total workload cost / accepted tasks

total workload cost = first attempts + paid retries
                    + paid fallback calls + other included costs

Accepted tasks must be defined by the same workload acceptance rule for every route. For an API-only comparison, total workload cost includes every billed request within the stated scope, including paid failures, retries, and fallback calls. A fully loaded comparison can also include tools, infrastructure, and human review, but it must use that same boundary for every route. If no task is accepted, report cost per successful task as unavailable rather than a finite number. The worked Route A and Route B example appears below.

Start with the rate card, then leave it behind

A token price is a useful input. It tells you what a billable unit costs under a particular provider, mode, and context tier. It does not tell you how many units a workflow will consume, how often it will retry, or whether the output will be accepted. Here is a small cross-provider snapshot from the AICostBudget canonical pricing Dataset. It is a rate-card comparison, not a ranking of model quality or task economics.

Prices are USD per million text tokens, Standard mode, at the stated base or short-context tier. This is a snapshot of Dataset records generated September 30, 2026, not live billing data. OpenAI charges a higher tier above 272,000 input tokens for the two GPT-6 models shown; Google caching has storage charges; cache writes can have separate charges. Cached input is a rate for eligible cache hits, not a blanket discount on all input. Provider, region, mode, and token shape can change the bill. Check the date and conditions before using any row in a budget.

Inspect the pricing Dataset and the official pages for GPT-6 Luna, GPT-6 Sol, Claude Haiku 4.5, and Gemini 3.5 Flash-Lite.

Luna is the lowest listed input and output rate in this four-row snapshot. That alone does not make it the cheapest model for your ticket queue, coding agent, or document pipeline. A retry resends some or all of the prompt and produces another output. A fallback may repeat the request on a different model. Failed tasks can still use billable resources. The price table supplies attempt rates; the workload supplies the attempt count and the successful-task denominator.

Define success before comparing models

A successful task is a workload-specific outcome, not a model leaderboard score or an HTTP 200. For structured extraction, success might mean that required fields parse and match a checked record. For coding, tests may need to pass and a reviewer may need to accept the change. For support, the ticket may need to be resolved without reopening. Write the rule before you run the comparison, apply it to both routes, and keep tasks that fail the rule in the original incoming-task count.

Be equally precise about the boundary of a task. One customer ticket can contain several model requests. One coding assignment can contain a planner call, tool calls, a repair pass, and a final answer. If you divide spend by API requests, those pieces become unrelated rows. Give them one parent task ID, record the model and token use of each attempt, and attach the final accepted or unresolved outcome to the parent.

Use provider billing records to distinguish an unbilled provider error from a validation failure that produced a billable response. The article's worked example counts model API spend only; do not compare it with an all-in route cost unless both routes use the same expanded boundary.

A worked comparison of calls and accepted tasks

Assume 10,000 incoming tasks a month. Route A uses a cheap first attempt at $0.004. Route B uses a different first attempt at $0.006. A fallback attempt costs $0.020 in either route. All attempt counts and acceptance rates below are illustrative assumptions, not benchmarked model results. The costs are examples too, not the prices of specific models in the rate card above. Your token mix and model choice would determine real attempt costs.

In this example, Route A makes 2,000 additional attempts: 1,000 on its primary and 1,000 on the fallback. It finishes with 9,000 accepted tasks. Route B makes 500 additional attempts: 400 on its primary and 100 on the same fallback. It finishes with 9,700 accepted tasks. Additional attempts here are counted across incoming tasks; they are not a claim that exactly that many distinct tasks retried.

Route A has a lower first-call price and even a slightly lower monthly API bill. Route B delivers 700 more accepted tasks for $0.40 more, so its API cost per accepted task is lower: about $0.00664 against $0.00711. The difference is not a universal savings claim. It follows only from the stated attempts, costs, and outcomes. Change any of them and the decision may reverse.

This is why reporting both cost per incoming task and cost per accepted task matters. The first tells finance what the workload spends for each job it receives. The second tells the product team what that spend buys under its acceptance rule. Neither should be silently substituted for the other.

The retry tax is bigger than one extra response

A failed attempt can have a large input side. An agent may resend its system prompt, tools, retrieved documents, conversation history, and the error from the previous attempt. Output can grow as the agent explains and repairs its own failure. A cache hit may soften repeated input cost, but eligibility depends on the provider and on whether the prefix stays stable. Switching models or rebuilding context can lose that benefit. This is why counting retries alone is insufficient: measure tokens per attempt, including cached and uncached input.

Fallback is a distinct decision. Some systems use it for provider errors or rate limits; others escalate after a quality gate rejects a usable response. Do not call every failure a model-quality problem. If the fallback only handles a transport error and the original request was unbilled, its economics differ from a route that pays for an invalid answer and then repeats the same context. The attempt log should say why escalation happened and whether the first call was charged.

For example, OpenRouter documents ordered model fallbacks for errors, while LiteLLM documents separate retry and fallback controls. Neither routing configuration tells you whether a final answer passed your own acceptance check. Link the attempt path to that outcome.

When the cheaper model really is cheaper

Suppose Route A's cheap first attempt still costs $0.004, but a simpler task mix allows it to accept 9,800 of 10,000 jobs without any paid retry or fallback. Its monthly API spend is $40, and its cost per accepted task is $40 / 9,800, or about $0.00408. That beats Route B's $0.00664 in the example. The cheaper model wins when its acceptance rate remains high enough, its attempts stay short, and escalations are rare. There is no rule that premium models are more economical.

A deterministic schema check can also make a smaller model practical. If it rejects malformed output cheaply, the remaining question is whether the retries it triggers buy enough accepted results. A low-cost classifier, short extraction, or routine rewrite may be an excellent fit for a low-rate model. Keep the successful workload slice separate from hard cases rather than averaging away where each route works.

Test the choice on your own workload

• Sample the task types and difficult cases your product actually receives. Preserve the same task set, prompt policy, and acceptance check for each candidate route.
• Record first-pass acceptance and final accepted or unresolved status. A benchmark score cannot stand in for this workload-specific result.
• Attach every paid call to a parent task ID. Keep primary retries, fallback calls, tool calls, cached input, uncached input, and output tokens visible by attempt.
• Compare monthly task volume, total spend, cost per incoming task, and cost per accepted task. Inspect the expensive tail and retry causes, not only the mean.
• Set a retry or spend cap and rerun the evaluation. A cap can reduce runaway cost, but it can also leave more tasks unresolved; measure both effects.

If you are considering a direct move between two models, the model-switch comparison can help price the same token workload. Treat that as a first pass: it holds attempt behavior fixed. The task-level retry calculation is the next step once you know how often each route needs more work.

Use the model-switch comparison for an equal-token price check. For current list prices and freshness, consult the model pricing comparison. Then return to the attempt log and accepted outcomes before choosing a production default.

If you only have planning assumptions, label them as assumptions. If you have traces, replace the estimates with observed attempt counts and provider charges. Repeat the calculation after prompt, routing, model, or cache-policy changes. The useful unit is the task your user asked you to complete, and the denominator is the work your team actually accepts.

Keep the outcome ledger stable while comparing routes. If a ticket is handed to a person, decide whether the task counts as completed by the combined workflow or unresolved by the AI route, and include the person's time if you choose the former. Do not drop abandoned jobs from one candidate's denominator while counting them for the other. A weekly report can show automated acceptance beside eventual resolution, with a separate cost boundary for each. If a route needs a person to finish its hardest tickets, that work belongs in the ledger too.

FAQ