Home / LLM API Pricing / Reducing costs / Batch processing

Batch API pricing

Why this is often the bigger lever

Prompt caching gets more attention, and for a chat product it usually deserves it. But caching only ever discounts input, and only the repeated prefix of it. A batch discount applies to input and output alike, at a flat rate, regardless of what your prompts look like.

For a generation-heavy workload that difference is decisive. Output typically costs four to six times input, so it dominates the bill, and caching cannot touch it. A summarisation job that reads short documents and writes long summaries might see caching cut 5% of its bill and batching cut 50%.

They are not alternatives. A nightly enrichment job with a large fixed system prompt can often take both, and where a provider allows them to stack the discounts multiply rather than add.

What it looks like on a real job

Bulk classification is the archetypal batch workload: high volume, tiny outputs, nobody waiting. At 5,000,000 items a month at 400 input and 20 output tokens each, on Claude Haiku 4.5:

Real-time

$2,500per month at list rates

Batched

$1,250at 50% off

Saved

$1,250$15,000 a year

Nothing about the model, the prompt or the output changed. The only difference is that the results arrive within hours instead of within seconds, which for a classification backfill is not a difference at all. Run your own numbers in the batch API savings calculator.

Which workloads qualify

The honest version of this question is not "what could tolerate delay in principle" but "what does no one look at immediately". Good candidates:

• Bulk classification and tagging — content moderation queues, ticket routing, catalogue enrichment.
• Offline summarisation — digests, meeting notes processed after the fact, document indexing.
• Dataset labelling — training-data generation, synthetic data, annotation passes.
• Embedding backfills — re-embedding a corpus after a model change.
• Evaluation runs — regression suites and benchmark sweeps, which are pure batch work.
• Scheduled reports — anything generated on a cron and read later.

What does not qualify: anything in a request path, anything a user triggers and waits for, and anything with an SLA measured in seconds. Splitting a workload is normal — many products batch their backfill and keep their live path real-time — and the calculator asks what share can move rather than assuming all of it.

Which providers offer it

Batch pricing is not universal, and it is worth checking before you architect around it. 18 models publish a batch discount; the providers offering one on at least some models are Alibaba, Anthropic, Google, Groq.

The providers with no published batch discount on any tracked model are Amazon, Baidu, ByteDance, Cohere, DeepSeek, Meta, Mistral, Moonshot AI, OpenAI, Perplexity, StepFun, Zhipu AI, xAI. On those, the levers are how prompt caching works, output limits, and tier selection. Full rate cards including batch columns are on the every LLM API provider pages and the compare LLM API pricing.

The engineering cost

Worth naming, because it is the reason teams skip a 50% discount. Moving to batch means holding request payloads somewhere, submitting jobs, polling for completion, handling partial failures and retries on individual items, and reconciling results back to whatever produced them. For a workload already running as a scheduled job, that is a modest change. For one currently running inline behind an HTTP handler, it is a genuine piece of work.

Compare it against the annual figure rather than the monthly one, since that is the horizon the work pays back over. At the volumes where batch matters, it usually pays back quickly.

Batch pricing questions