Self-hosting cost is a utilization question
An API bills per token, so cost grows in step with usage. A rented GPU bills per hour, so cost is fixed whether the model serves one request or keeps every GPU busy. The cheaper option therefore depends less on the hourly GPU price than on how much of each paid hour your workload actually uses.
How the calculator works
API monthly cost = requests x (input tokens x input price + output tokens x output price) / 1,000,000
self-hosted monthly cost = GPU $/hour x GPUs x hours online + other monthly cost
seconds per request = input tokens / prefill tok/s + output tokens / decode tok/s
GPU utilization = requests x seconds per request / (hours online x 3,600)
break-even requests = self-hosted monthly cost / API cost per requestThroughput is the aggregate rate of the whole deployment at realistic concurrency, not the speed a single user sees. If the workload needs more than 100% of the online GPU time, the calculator flags it instead of reporting a cost the deployment cannot deliver.
Worked example: one always-on GPU vs GPT-5.4 mini
Using the calculator defaults: one GPU at $3/hour online 730 hours a month ($2,190), 5,000 prefill and 1,500 decode tokens per second, and requests of 1,500 input and 500 output tokens. The API side uses OpenAI GPT-5.4 mini at $0.75/1M input and $4.5/1M output. The GPU and throughput figures are illustrative assumptions, not measurements.
| Requests/mo | API | Self-hosted | GPU use | Cheaper |
|---|---|---|---|---|
| 10,000 | $34 | $2,190 | 0.2% | API |
| 100,000 | $338 | $2,190 | 2.4% | API |
| 1,000,000 | $3,375 | $2,190 | 24% | Self-hosted |
| 4,000,000 | $13,500 | $2,190 | 96% | Self-hosted |
Change the API model, GPU price or throughput in the calculator above to see where the break-even point moves for your own numbers.
When self-hosting can make sense
- Steady, predictable volume that keeps GPUs busy most of the day.
- Data that must stay inside your own infrastructure or region.
- A fine-tuned or specialized open-weight model that no API offers.
- Latency or throughput guarantees you need to control yourself.
Spiky or low traffic, a need for frontier-model quality, or a small team without inference operations experience usually favor an API. To find the lowest per-token option first, see the cheapest LLM API comparison.
Assumptions and limitations
The calculator assumes a constant request mix and an evenly loaded deployment. It does not model traffic peaks, autoscaling, redundancy, engineering time, storage, egress, batch or cached API discounts, or quality differences between models. Add known extra costs to the other monthly cost field. API prices come from verified Standard price records documented in the AI API Pricing Dataset.
Related cost tools
Estimate request token counts with the Token Calculator, or compare two API models for the same workload with the Model Switch Cost Calculator. For break-even request counts against eight API price tags, see when self-hosting an LLM pays off.
Frequently asked questions
Is it cheaper to self-host an LLM than to use an API?
Only when the GPUs stay busy. A rented GPU costs the same whether it serves one request or a million, so at low volume the per-token API price is usually far cheaper. Self-hosting wins when steady volume keeps utilization high enough that the fixed GPU bill is lower than the API bill for the same tokens.
How is the break-even point calculated?
The calculator divides the monthly self-hosted cost (GPU price x GPUs x hours online, plus any other monthly cost you enter) by the API cost of one request. That gives the monthly request volume where both options cost the same. If that volume is higher than your deployment can serve, the API stays cheaper at every load the deployment can handle.
What throughput should I enter?
Enter the aggregate tokens per second your whole deployment sustains under realistic concurrency, measured with your model, quantization, context length and serving stack. Prefill (input) is usually much faster than decode (output), so they are separate inputs. The defaults are placeholders, not benchmarks.
What does GPU utilization mean here?
It is the share of online GPU hours needed to process the workload at the throughput you entered. 5% utilization means the deployment is idle 95% of the time you are paying for it. Real deployments also need headroom for traffic peaks, so planning near 100% is not realistic.
What costs does this calculator leave out?
Engineering and on-call time, redundancy and failover, storage, networking and egress, autoscaling delays, model quality differences, and any GPU discounts or commitments are not modeled unless you add them to the other monthly cost field or the GPU hourly price.
Where do the API prices come from?
API prices are the verified Standard input and output prices from the AICostBudget pricing dataset, each linked to the provider's official pricing page and verification date. GPU prices and throughput are your own assumptions; AICostBudget does not verify them.
Which API model should I compare against?
Choose an API model with similar quality for your task to the open-weight model you plan to host. Comparing a small self-hosted model with a frontier API model, or the reverse, compares different products rather than different ways to pay.