26 AI API billing terms defined in plain English: tokens, cached input, cache writes, batch pricing, long-context tiers and cost per task, with dated data.
This glossary defines 26 AI API billing terms. Three move a bill the most: cached input, which costs a median 10% of standard input; five-minute cache writes, a median 125% of input; and batch pricing, a median 50% input discount. Figures come from 50 current models, verified Oct 21, 2026.
A token is the unit an AI model reads and writes: a word, part of a word, punctuation, or a piece of other content. Each model's tokenizer splits the same text differently, so token counts vary between models.
Providers measure and bill text usage in tokens, separately for input and output. Count tokens with the model's own tokenizer before estimating cost.
Input tokens are everything sent to the model in a request: system instructions, the user message, conversation history, retrieved documents, tool definitions, and examples.
Billed at the model's input price per million tokens. Repeated prefixes can be billed at the lower cached-input rate when prompt caching applies.
Output tokens are the tokens a model generates in its response, including reasoning tokens that a provider bills as output even when they are not shown.
Billed at the model's output price per million tokens. Limit response length and reasoning effort to control them.
Reasoning tokens, also called thinking tokens, are tokens a reasoning model generates while working through a problem before its final answer. They may be hidden or summarized instead of returned in full.
OpenAI and Anthropic bill them as output tokens, so a short visible answer can still produce a large output bill. For other providers, check their documentation before estimating.
Context window is the maximum number of tokens a model can work with in a single request. Long prompts, conversation history, and retrieved documents all draw on it.
The window itself is not a charge, but filling it is: every token sent is billed as input, and some models switch to a long-context tier above a threshold.
Price per million tokens is the standard unit providers use to publish token prices: the cost of processing one million input, cached-input, or output tokens with a given model.
Divide your token count by one million and multiply by the published price, separately for each kind of token.
Input:output ratio is how many input tokens a workload sends for each output token it receives. Retrieval and long-document tasks are input-heavy, while drafting and code generation are output-heavy.
Because input and output are priced differently, the ratio decides which model is cheaper for a workload. Compare pages show the break-even ratio between two models.
Across 56 current text models we track, output tokens cost a median 5.0× the input price (range 1.0×–8.3×); output is more expensive for 95% of them, with prices verified Jul 7, 2026 to Oct 21, 2026.
Prompt caching lets a provider reuse the processing of a repeated prompt prefix, such as a long system prompt or shared document, so later requests that start the same way are billed at a lower cached-input rate.
Some providers cache automatically; others require you to mark cacheable content and may charge a cache-write fee when it is first stored.
Cached input is input that the provider serves from its prompt cache instead of processing it again. It is billed per token at the cached-input rate, which is lower than the standard input rate.
Only the repeated prefix qualifies; new content in each request is still billed at the standard input price.
As of Oct 21, 2026, 46 of the 50 current models with a pricing page list a cached-input price; the median is 10% of the standard input price (range 2%–25%).
Cache write is the charge some providers apply when content is first stored in the prompt cache. It is billed per token, can cost more than standard input, and may depend on how long the entry is kept.
A cache pays off only when later cache reads save more than the write cost; short-lived or rarely reused prefixes can cost more than not caching.
As of Oct 21, 2026, 18 current models list a cache-write price. Median cache-write price as a share of standard input: 125% for five-minute writes, 200% for one-hour writes, 250% for twelve-hour writes, 125% for writes with no stated lifetime.
Cache lifetime is how long a cached prompt stays available before it expires. Providers that offer more than one lifetime charge a different cache-write price for each.
Longer lifetimes cost more to write but keep the cache warm across slower traffic; choose the shortest lifetime that still covers the gap between requests.
As of Oct 21, 2026, current models publish cache writes with these lifetimes: five-minute (11 models), one-hour (11 models), twelve-hour (7 models), no stated lifetime (7 models).
Cache hit rate is the share of input tokens served from the prompt cache instead of being processed at the standard rate. A higher rate lowers the effective input price of a workload.
Cache writes, expirations, and prompts whose beginnings change between requests all lower it.
Batch API pricing is the discounted rate providers charge for requests submitted as an asynchronous batch that may take longer to complete, instead of returning each result immediately.
Use it for work that does not need a live answer, such as evaluations, backfills, and bulk classification.
As of Oct 21, 2026, 40 current models publish batch prices; the median batch discount is 50% on input and 50% on output versus standard prices.
Long-context tier is a different price that applies when a request's prompt exceeds a published token threshold. With whole-request pricing, the tier applies to every token in that request, not only to tokens above the threshold.
Keep prompts under the threshold where you can, or compare models whose standard tier covers your prompt length.
As of Oct 21, 2026, 17 current models price long prompts in a separate tier; the lowest published threshold is 100,000 prompt tokens.
Free tier is a provider allowance that lets you call an API without paying, often with lower rate limits or a restricted list of models. Allowances and data-use terms differ by provider and change often.
Treat a free tier as a prototype budget, not a production plan; check the official limits before relying on it.
Promotional pricing is a temporary discount a provider applies to a model, often at launch, before the standard list price takes effect.
AICostBudget stores promotional and list charges separately and does not use a promotion without a dated end in default calculations.
Storage charges are fees for keeping data on the provider's side over time, such as cached content or files, billed per unit of data per period instead of per request.
They accrue while the data is kept, so delete or let unused stored content expire.
As of Oct 21, 2026, 3 records in the published dataset list a storage charge.
Per-page pricing is how document and OCR APIs commonly bill: a fixed price for each processed page, usually quoted per thousand pages, regardless of how many tokens the page contains.
Estimate by page count, not by text length; multi-page scans and rejected pages still count.
As of Oct 21, 2026, 6 records in the published dataset list a per-page document charge.
Session-duration pricing bills realtime or voice sessions by connected time, per minute or per hour, instead of by the tokens exchanged.
Idle connected time is billed too, so close sessions promptly.
As of Oct 21, 2026, 5 records in the published dataset list a session-duration charge.
Audio tokens are the token units speech-capable models use for spoken input and output. They are priced separately from text tokens.
Convert audio length to tokens with the provider's published rate before comparing it with text pricing.
As of Oct 21, 2026, 6 records in the published dataset price audio per million tokens.
Cost per request is what a single API call costs: input tokens times the input price, plus output tokens times the output price, plus any cached-input, cache-write, or tool charges.
Multiply it by monthly request volume for a first budget, then add retries and growth.
Cost per successful task is the total spend needed to finish one unit of work, including retries, fallback calls, and failed attempts, divided by the tasks that actually succeed.
A model with a lower token price can still cost more per task if it fails more often.
Retry and fallback cost is the extra spend from calls that fail, time out, or are rejected and must be repeated, either on the same model or on a fallback model.
Every attempt is billed, including the ones that produce no usable answer.
List price vs effective cost is the gap between the published price per token and what a workload actually pays after caching, batch discounts, retries, long-context tiers, and free allowances are applied.
Budget on effective cost, and recheck it whenever traffic patterns or prompts change.
Price change event is a verified change to a model's published pricing or lifecycle, recorded with its scheduled, effective, detected, and verified dates in the price change log.
Recheck budgets and model choices when a model you use has a new event.
17 verified price change events were recorded in the 90 days to Oct 21, 2026.
Last verified is the date a price was most recently checked against the provider's official pricing source. It records when the check happened, not a promise that the price is still current.
Prefer recently verified prices, and confirm with the provider's official page before committing a budget.
Dated figures on this page are computed from the published AI API pricing dataset, last verified Oct 21, 2026 (How we verify). Browse model pricing pages, or compare two models on the head-to-head pages.