Batch API savings calculator
If nobody is waiting on the answer, you are probably paying twice what you need to. Batch APIs take a flat 50% off both input and output in exchange for giving up real-time latency.
How this is calculated
The share of your traffic you mark as delay-tolerant is priced at the discounted batch rate; the rest stays at the standard real-time rate. The headline figure is the difference between that mix and running everything in real time, so it is the marginal saving from adopting batch rather than your total bill.
The percentage field is the honest part of this calculation. Very few products can batch everything, and the instinct is to assume more of the workload is delay-tolerant than really is. Count only the requests where no user is blocked on the result.
Why batch is often the bigger lever than caching
Prompt caching gets more attention, but it only ever discounts input, and only the repeated prefix of it. A batch discount applies to both input and output at a flat rate, regardless of what your prompts look like. For a generation-heavy workload, where output is four to six times the input rate and dominates the bill, that difference is decisive.
The two are not alternatives. A nightly enrichment job with a large fixed system prompt can often take both, and where a provider allows them to stack the discounts multiply: half of a tenth is a twentieth. Size each one separately rather than guessing at the combined figure.
Batch pricing questions
What is a batch API?
An asynchronous queue. Instead of holding a connection open and waiting for a response, you submit a job and collect the results later, typically within a few hours. The provider gets to schedule the work when it has spare capacity, and passes some of that saving back as a discount.
How much do batch APIs save?
A flat 50% off both input and output on every model tracked here that offers one, across 4 providers. Unlike prompt caching, it applies to output too, which is why it is often the larger saving for generation-heavy work.
Which workloads can use a batch API?
Anything where nobody is waiting on the answer: bulk classification and tagging, offline summarisation, dataset labelling and enrichment, embedding backfills, evaluation runs, and nightly report generation. Anything with a user watching a spinner cannot use it.
What is the catch with batch processing?
Latency, and the engineering around it. Jobs typically complete within hours rather than seconds, and you need somewhere to put the request payloads, a way to poll for completion, and handling for partial failures. For a workload already running as a scheduled job that is minor; for one currently running inline, it is a real change.
Can I combine batch pricing with prompt caching?
Sometimes, and where both apply they multiply rather than add. Check the provider's documentation, then size each lever separately: this page for batch, the prompt caching savings calculator for the cache side.