Prompt caching
Prompt caching discounts input tokens the provider has already processed for you. 27 of the 60 models tracked here publish a cached rate, at a median around 90% off standard input. It is the only major cost lever that changes nothing about your output.
It matches on a prefix, which is the detail everything else follows from: if anything varies near the top of your prompt, nothing after it caches.
How prefix matching works
When you send a request, the provider compares the opening of your prompt against what it recently processed. The longest matching stretch from the start is served from cache; everything from the first difference onward is computed fresh and billed at the standard rate.
That is why it is a prefix and not a search. A cache does not find your system prompt somewhere in the middle of a request and discount it. It reads from the first token forward and stops at the first thing that changed. Two requests that are 95% identical but differ in the second sentence share almost no cacheable prefix at all.
What breaks a cache hit
Nearly every zero-hit-rate report comes down to one of these, and all of them are ordering problems rather than caching problems:
- A timestamp or date in the system prompt. "Today is 16 August 2026, 14:32" at the top of a prompt invalidates the cache on every single call.
- A user or session identifier near the start. Personalising the opening line means every user gets their own cache entry, and none of them share.
- Non-deterministic serialisation. Tool definitions or JSON context assembled from a hash map can come out in a different order between calls. Same content, different bytes, no hit.
- Rebuilding the prompt from templates per request. Minor whitespace or punctuation differences count as differences.
The fix in every case is the same: everything stable first, byte-identical; everything variable last. Treat the opening block of your prompt as frozen and append rather than interleave.
What a realistic hit rate looks like
Two separate numbers decide your saving, and conflating them is why people overestimate caching. Cacheable prefix is how much of each request could ever be cached: the stable opening, before anything that varies. Hit rate is how often that prefix is still in the cache when the request arrives. Your effective discount is the product of the two, not either one.
A chat product with a large fixed system prompt and tool definitions might have 80-95% cacheable prefix and a high hit rate during an active session, which is close to the best case. A batch of one-off requests scattered across users and topics might have the same 90% cacheable prefix and a hit rate near zero, because nothing arrives close enough together. Bursty traffic caches well; evenly spread traffic often does not.
Size it properly with the prompt caching savings calculator, which asks for both numbers separately rather than assuming a perfect hit rate.
The deepest cached-input discounts tracked
Most providers land near 90% off input, but the range is wider than that suggests, and the outliers are worth knowing about if your workload is cache-heavy enough to choose a model on this basis.
| Model | Provider | Standard input | Cached input | Discount |
|---|---|---|---|---|
| DeepSeek V4 Pro | DeepSeek | $0.435 | $0.003625 | 99% |
| DeepSeek V4 Flash | DeepSeek | $0.14 | $0.0028 | 98% |
| GPT-5.6 Luna | OpenAI | $0.20 | $0.02 | 90% |
| GPT-5.4 mini | OpenAI | $0.75 | $0.075 | 90% |
| GPT-5.4 nano | OpenAI | $0.20 | $0.02 | 90% |
| Claude Sonnet 4.6 | Anthropic | $3.00 | $0.30 | 90% |
| Gemini 3.6 Flash | $1.50 | $0.15 | 90% | |
| Gemini 3.5 Flash | $1.50 | $0.15 | 90% |
DeepSeek is the standout: its cache-hit rate is under 1% of standard input, an order of magnitude steeper than the industry norm of around 10%. For an application that resends a large static prompt on every call, that can make a nominally mid-priced model the cheapest option available.
When caching is not worth the work
It is a genuine engineering change, not a config flag, so it is worth knowing when to skip it. Caching does little when prompts are short (there is no large prefix to save), when the workload is generation-heavy (output is untouched by it), when requests are spread thinly enough that entries expire between them, or when the model you use does not publish a cached rate at all — which is the case for 33 of the 60 models tracked here.
In those cases the next lever is usually how batch processing pricing works, which discounts output as well and does not care how your prompt is structured.
Prompt caching questions
What is prompt caching?
A discount on input tokens the provider has processed for you recently. When a request starts with the same prefix as a previous one, the provider serves that prefix from a cache instead of recomputing it, and bills it at a much lower rate. It changes nothing about the output: same model, same response, lower input cost.
How much does prompt caching save?
A median of about 90% off the standard input rate across the 27 models that publish one. The deepest discount tracked here is DeepSeek's DeepSeek V4 Pro, where a cache hit costs $0.003625 against $0.435 standard. What you actually save depends on your hit rate and on how much of each request is cacheable prefix.
Why is my cache hit rate zero?
Almost always prompt ordering. Caching matches on a prefix, so a single varying token near the top of the prompt — a timestamp, a session ID, a user name, a randomly ordered tool list — invalidates everything after it. Move volatile content to the end and keep the opening block byte-identical between calls.
Does prompt caching reduce output costs?
No. Caching only ever applies to input. Output is generated fresh every time and billed at the full rate, which is why a workload with short prompts and long completions sees very little benefit no matter how good its hit rate is.
How long does a cached prompt last?
Typically minutes rather than hours, and it varies by provider. That makes traffic shape matter as much as prompt shape: requests concentrated into active sessions cache well, while the same daily volume spread thinly may never hit at all.
Which models support prompt caching?
27 of the 60 models tracked here publish a cached-input rate. 33 do not, so on those the lever is unavailable and the standard input rate is what you pay.