Self-Hosting an LLM? At 100,000 Requests a Month, Even Claude Opus 5.5 Is Cheaper Than Your GPU

The $2,190 you pay before the first token

Somebody on your team has done the napkin math. Open-weight models are good now, GPUs are easy to rent, and the API invoice keeps growing. Why pay per token when you could pay a flat rate and own the box?

Here is the part the napkin skips. A rented GPU starts billing the moment it boots. At $3 an hour, always on, that is $2,190 every month, and that number does not care whether the GPU answered ten requests or four million. An API is the opposite: zero requests, zero bill.

So the question is not "which is cheaper per token." It is "how many requests do I need to send before a flat $2,190 beats paying per call." Below that line, every dollar of GPU rent you pay is a dollar you handed over for idle silicon.

To make it concrete, every number in this post uses one workload: requests of 1,500 input tokens and 500 output tokens, priced at each provider's Standard list rate. On the GPU side we use the defaults from our self-hosted calculator: one GPU at $3 an hour, online 730 hours a month, sustaining 5,000 prefill and 1,500 decode tokens per second across all concurrent requests. Those GPU figures are assumptions for illustration, not benchmarks of any model.

Read that chart twice. At 100,000 requests a month, the always-on GPU costs more than running every one of those requests through Claude Opus 5.5, one of the most expensive models on the list. It costs 54.75 times what the same traffic costs on Claude Haiku 5.5. And the GPU is busy for 2.4% of the hours you pay for. The other 97.6% is rent on a machine that is waiting.

The request count where one GPU starts to win

The break-even is one division: monthly GPU cost ÷ API cost of one request. Above that request count, the flat rent is cheaper. Below it, the API is. There is a second number that matters just as much: how many requests the GPU can actually serve in a month.

API cost per request   = (1,500 × input price + 500 × output price) ÷ 1,000,000
break-even requests    = $2,190 ÷ API cost per request

GPU time per request   = 1,500 ÷ 5,000 + 500 ÷ 1,500 = 0.633 seconds
monthly capacity       = 730 h × 3,600 s ÷ 0.633 s = 4,149,473 requests

Two things jump out. Against frontier-priced APIs the line is low: 136,875 requests a month against Opus 5.5 is roughly 4,500 a day. If you really can replace Opus-class output with a model that fits on one GPU, that is a volume plenty of products reach.

Against the budget tier the line climbs fast, and for two models it leaves the chart entirely. To beat DeepSeek V4.1 Flash the GPU has to stay half busy every hour of every day, nights and weekends included. To beat Claude Haiku 5.5 it would need 5,475,000 requests, about 1.3 million more than it can serve. That is not a slow road to savings. There is no road.

Why a cheap API can't be beaten by one GPU

Flip the math around and look at what one request costs you on the GPU at different volumes. The rent is fixed, so the cost per request falls as traffic grows. But it can only fall until the GPU is full.

The floor is $0.528 per 1,000 requests. Mistral Small 4 charges $0.525 for the same 1,000 requests and Claude Haiku 5.5 charges $0.40. So even a GPU running at 100%, with no spare room for a traffic spike and no engineer's time counted, loses to Mistral Small 4 by three tenths of a cent per thousand requests. A faster serving stack or a cheaper GPU can move that floor. A busier month cannot.

This is why the first check before any self-hosting plan should be the cheapest API that already meets your quality bar. Our cheapest LLM API comparison ranks the budget tier for a small workload. If one of those models handles your task, the GPU has to beat a very low price, and at one-GPU scale it usually can't.

Your real ceiling is lower than 100%

Nobody runs a production GPU at 100%. Traffic arrives in waves, and if the deployment is full at the peak, requests queue and latency explodes. Planning for 70% leaves room for a busy hour. That lowers the useful capacity from 4,149,473 to about 2.9 million requests a month. DeepSeek V4.1 Flash still fits under that line, but its 2,085,715-request break-even already uses about 72% of the room you planned for.

Then there is the bill that never shows up in a GPU price list: the person who keeps it running. Model updates, driver issues, a 3 a.m. out-of-memory crash. Here is what happens if you put $1,000 a month on that, say 10 hours of an engineer at $100 an hour. Pick your own number; this one is only an example.

Every break-even rises by 46%, because $1,000 is 46% of the GPU rent. On the budget side that is enough to push DeepSeek V4.1 Flash past the 70% planning ceiling. If your plan only works at 100% load with free engineering, it doesn't work.

The setting that cuts your break-even by 70%

Most napkin math assumes the GPU runs 24/7. If your traffic is an internal tool, a back-office pipeline or anything that only happens in working hours, it doesn't have to. Run it 10 hours a day on 22 workdays and you pay for 220 hours instead of 730.

That is a 70% lower break-even from changing one input. It is also the cheapest experiment you can run: if a scheduled GPU can't beat the API during the hours your traffic actually arrives, an always-on one won't either. The fine print is real, though. Every cold start pays to load model weights, on-demand GPUs are not always available the moment you ask, and anything customer-facing at night still needs a home, often the same API you were trying to leave.

In the self-hosted LLM vs API calculator, this is the "Hours online per month" field. Set it to the hours your traffic really needs, and watch the break-even line move.

When self-hosting does pay

None of this says self-hosting is a mistake. It says the GPU is a bet on volume, and you should know the size of the bet before you place it. The numbers favor running your own model when:

• You are replacing an expensive API. Against Opus 5.5 or GPT-6.1 Sol pricing, one GPU breaks even at a few hundred thousand requests a month, and an open-weight model that fits on that GPU may be good enough for narrow tasks like classification or extraction.
• Your traffic is steady. A flat line all day keeps utilization high without overloading the peak. Spiky traffic is what makes APIs cheap.
• Data has to stay on your own infrastructure. Then the comparison is not cost alone, and the break-even tells you what that control costs per month.
• You run a fine-tuned or specialized model that no API sells.

One warning about the first point. Comparing a self-hosted model with Opus 5.5 only makes sense if your model passes the same test on your own tasks. A small model that needs retries or human fixes is not a cheaper Opus; it is a different product with a hidden bill. The fair comparison is against the API of similar quality for your job, which often sits in the budget tier, which brings us back to the bottom of the chart.

Check your own numbers in two minutes

The example GPU above is made up on purpose. Your quote, your model and your serving stack will give different numbers, and the only throughput figure worth trusting is one you measured under realistic concurrency. Here is the quickest way to get an answer you can defend in a budget meeting:

• Choose the API model you would actually replace, one with similar quality for your task.
• Enter your average input and output tokens per request and your monthly request count. Our Token Calculator helps if you don't know them yet.
• Enter your GPU quote per hour, the number of GPUs, and the hours the deployment must be online.
• Enter prefill and decode tokens per second from a load test of your own model, not a single-user speed test.
• Add engineering and other monthly costs, then read the GPU utilization and the break-even request count.

If the break-even sits well under 70% of capacity with your real costs in it, self-hosting is worth a pilot. If it sits above, the free self-hosted vs API calculator just saved you a GPU contract. Need token counts first? Start with the Token Calculator.

FAQ