Self-hosting an LLM is a utilisation bet, not a savings plan
The comparison people make is dollars per GPU-hour against dollars per million tokens. Those units aren't comparable until you decide how busy the GPU will be — and that decision is the whole argument.
Self-hosting an open-weight model looks obviously cheaper right up until you do the arithmetic properly. The instinct is sound: open weights are free, GPU rental is a known quantity, and paying per token forever feels like renting when you could own. The instinct is also incomplete, because it compares a fixed cost to a variable one without asking the question that connects them.
That question is: how busy is the GPU?
An API bills you for tokens you actually generated. A rented GPU bills you for wall-clock time, whether it is saturated or sitting idle at three in the morning. So the comparison is not "cheap per token versus expensive per token." It is "you pay for what you use versus you pay for what you reserved." Everything else follows from that.
The break-even, with the assumptions stated
Take an H100-class GPU at $2.50 per hour — a mid-market rate at the time of writing; you can substitute your own in the GPU rental calculator. Running continuously, that is $1,825 per month, fixed, before anyone sends it a request.
Now the API side. Using a 3:1 input-to-output token mix, our dataset gives a blended rate of $1.695 per million tokens for a mid-priced model like Doubao Seed 2.1 Pro, and $0.175 for a cheaper one like DeepSeek V4 Flash.
Divide the fixed monthly cost by the variable rate and you get the break-even volume:
- To beat the mid-priced API, the GPU must serve about 1,077 million tokens per month.
- To beat the cheap API, it must serve about 10,429 million tokens per month.
Translating volume into utilisation
Raw token counts are hard to reason about, so convert them into the thing that actually matters. A single GPU of this class, serving a mid-size open-weight model with decent batching, sustains somewhere around 900 tokens per second in aggregate across concurrent requests. Over a month that is roughly 2,365 million tokens of theoretical capacity.
So the break-evens become utilisation targets:
- Beating the mid-priced API needs roughly 46% sustained utilisation.
- Beating the cheap API needs roughly 441%.
And sustained is the load-bearing word. It means around the clock, every day, including weekends and the hours when your users are asleep. A workload that saturates a GPU during business hours and idles for the other sixteen is running at well under 40% utilisation no matter how impressive the peak looks.
Most teams that model self-hosting model the peak. The bill is determined by the trough.
What the calculation leaves out, and it is not small
The break-evens above are generous to self-hosting, because they count only the GPU. The honest total includes:
- Engineering time. Building and operating an inference stack — batching, quantisation, KV cache management, autoscaling, model loading, observability — is not a weekend. It is an ongoing responsibility with an on-call rotation attached.
- Redundancy. One GPU is one point of failure. Production availability means at least two, which roughly doubles the fixed cost and roughly doubles the break-even.
- Headroom for spikes. Capacity sized to average load falls over at peak. Capacity sized to peak is idle at average, which lowers your utilisation, which is the exact metric the whole case depends on.
- Model upgrades. An API silently improves under you. Self-hosted weights improve when you do the migration, re-benchmark, and re-tune the serving config.
Fold those in and the practical break-even sits materially above the arithmetic one. A reasonable rule of thumb is to double the naive figure before believing it.
When self-hosting is right anyway
None of this says don't self-host. It says don't self-host for cost reasons below the break-even. There are good reasons that have nothing to do with the token price, and they hold at any volume:
- Data residency and regulation. If the data cannot leave your infrastructure, the API comparison is moot.
- Version stability. A self-hosted model does not change under you. For anything requiring reproducibility or a validated pipeline, this alone can justify the cost.
- No third-party rate limits or deprecations. You control the capacity and the retirement schedule.
- Genuine fine-tuning. Full control of weights, not just an adapter behind someone else's API.
- Very high, very steady volume. If you are comfortably past the break-even with steady load, the economics do favour you — that is exactly what the break-even means.
The pragmatic path
The sequencing that works for most teams is dull and correct: start on the API, measure real sustained token volume for a quarter, then compute your own break-even from observed numbers rather than projected ones.
This costs you nothing. Per-token pricing means the API is cheap precisely while volume is low, which is exactly the period when you don't yet know your volume. You buy real data with money you would have spent anyway, and you make the capital decision with evidence. The opposite order — provisioning GPUs against a forecast — means paying for idle hardware while you find out the forecast was wrong.
The short version
- At $2.50/hour, a GPU costs $1,825/month whether busy or idle.
- Break-even against a mid-priced API is around 1,077M tokens/month — roughly 46% sustained utilisation on one GPU.
- Add redundancy, engineering time, and spike headroom, and the real break-even is roughly double the naive one.
- Residency, version stability, and control are better reasons to self-host than cost, and they apply at any volume.
- Start on the API, measure for a quarter, then decide from observed data.
Run it with your own GPU rate and throughput in the GPU rental cost calculator, and price the API side of the comparison across all 60 models on the comparison table.
Frequently asked questions
Is self-hosting an LLM cheaper than using an API?
Only above a break-even that depends on utilisation. At a $2.50/hour GPU ($1,825/month) you must serve roughly 1,077 million tokens per month to beat a mid-priced API, and about 10,429 million to beat a cheap one. Below that, the API is cheaper because you are not paying for idle hardware.
What utilisation do I need for self-hosting to pay off?
On the assumptions in this article — one GPU sustaining about 900 tokens per second — beating a mid-priced API needs roughly 46% sustained utilisation, and beating a cheap API needs around 441%. Sustained means around the clock, not during business hours.
Are there non-cost reasons to self-host?
Yes, and they are usually the better reasons: data residency and regulatory requirements, guaranteed model version stability, no dependency on a third party's rate limits or deprecation schedule, and the ability to fine-tune weights you control. These justify self-hosting at any volume. Cost alone rarely does below the break-even.
Does the GPU cost include everything?
No. The hourly rate is the largest line but not the only one. Engineering time to build and operate the serving stack, on-call coverage, redundancy for availability, storage and egress, and idle capacity held for traffic spikes all sit on top. A realistic total is meaningfully above the raw GPU-hour figure.
How this was produced. Every price in this article is read at build time from the dataset behind the comparison table — 60 models across 17 providers, each rate taken from the provider's own published pricing page. Nothing is quoted from a secondary source or a third-party aggregator. The methodology page sets out how rates are checked and what is deliberately excluded, and the underlying numbers are downloadable as JSON and CSV if you want to redo any of the arithmetic yourself.
Analysis and judgements are the author's own. Published list prices change without notice; confirm against the provider before making a billing commitment.