Prompt caching savings calculator
Caching is usually the single largest cost lever available on an LLM workload, and the one most people size wrong by assuming a perfect hit rate. Enter your real numbers and see what the discount is actually worth.
How this is calculated
Two of your inputs do different jobs and are easy to conflate. Cacheable prefix is the share of each request that could ever be cached: the stable opening of your prompt, before anything that varies per user or per turn. Hit rate is how often that prefix is actually still in the cache when the request arrives. Multiply them and you get the share of input tokens billed at the cached rate; everything else bills at the standard rate, and output is unaffected either way.
The calculator compares that against the same workload with caching switched off, so the figure it reports is the marginal saving from enabling caching rather than the total bill. For the total, use the API cost calculator.
Getting your hit rate up
The mechanic that matters most is ordering. Caching matches on a prefix, so anything that varies must come after everything that does not. A timestamp, a user ID, or a session variable placed near the top of a system prompt invalidates every token after it and can silently take a workload from a 90% hit rate to zero. Move volatile content to the end of the prompt, keep the stable block byte-identical between calls, and avoid regenerating tool definitions in a non-deterministic order.
Burstiness matters too. Cache entries expire on the order of minutes on most providers, so traffic concentrated into active sessions caches well while the same volume spread thinly across a day may not hit at all. If your traffic is bursty and your prefix is large, caching is close to free money. If it is uniform and your prompts are short, it may not be worth the implementation effort.
Prompt caching questions
What is prompt caching?
A discount on input tokens the provider has already processed for you recently. When a request starts with the same prefix as a previous one, that prefix is served from cache and billed at a much lower rate. Caching applies to the prefix only, so it works when the stable part of your prompt comes first and the variable part comes last.
How much does prompt caching save?
The median discount across the 27 models offering it is about 90% off the standard input rate. The deepest is DeepSeek's DeepSeek V4 Pro, where cache hits cost $0.003625 against $0.435 standard, a 99% cut. What you actually save depends on your hit rate and on how much of each request is cacheable prefix.
What is a realistic cache hit rate?
For a chat product with a fixed system prompt and tool definitions, 80-95% of input tokens can be cacheable prefix, with a high hit rate during active sessions. For one-off requests scattered across users and topics, it can be near zero. Cache entries also expire, typically in minutes, so traffic that arrives in bursts caches far better than traffic that trickles.
Does caching reduce output token costs?
No. Caching only ever applies to input. Output is generated fresh every time and is billed at the full rate, which is why a workload with short prompts and long completions sees very little benefit from caching no matter how good its hit rate is.
Can I stack caching with a batch discount?
On some providers, yes, and the two multiply rather than add. Where both apply, run the numbers through this calculator and the batch API savings calculator in sequence. Check the provider's own documentation before assuming they stack.