LLM pricing glossary
Every term used to bill an LLM API, defined once and in plain language. Where a term has a page of its own on this site, the definition hands off to it rather than repeating it. Figures verified 2026-08-16 across 60 models and 17 providers.
Token
The unit every LLM API bills in. Roughly four characters of English, or about 1.3 tokens per word, so one million tokens is around 750,000 words.
Code, non-English text and heavily formatted content tokenise less efficiently, sometimes much less. Model families use different tokenisers, so identical text can produce counts 10-35% apart depending on where you send it.
Estimate tokens from real text
Input tokens
Everything you send in a request: the system prompt, conversation history, retrieved documents, tool definitions and the user's message. Also called prompt tokens.
They are processed in parallel, which is why they cost less than output.
Output tokens
Everything the model generates back, also called completion tokens. Billed at a higher rate than input on nearly every model.
Each output token requires its own sequential forward pass through the model, so the premium reflects a genuinely more expensive operation rather than a pricing choice.
Cost per million tokens
The unit every provider quotes prices in, written as separate input and output rates. Often abbreviated per MTok.
Across the 60 models tracked here, input runs $0.03 to $30.00 per million, a 1000x spread.
Prompt caching
A discount on input tokens the provider has already processed for you recently. It matches on the prefix of a request, so anything that varies must come after everything that stays the same.
27 of 60 tracked models publish a cached rate, at a median around 90% off standard input. It never applies to output.
Cache hit rate
How often a cacheable prefix is still in the provider's cache when a request arrives. Distinct from how much of a request is cacheable in the first place.
Your effective discount is the product of the two, not either one alone. Conflating them is the most common reason caching savings get overestimated.
Batch API
An asynchronous queue: you submit work and collect results later, usually within hours, in exchange for a discount. The eligibility test is whether anyone is waiting on the answer.
18 tracked models offer one, almost always at a flat 50% off both input and output. Unlike caching, it discounts output too.
Blended rate
A single figure combining input and output rates at an assumed mix. This site assumes three input tokens per output token, which is closer to real chat and retrieval traffic than either rate alone.
It is a ranking convenience, not a law. If your own ratio is far from 3:1, price your actual mix rather than trusting the blend.
The market ranked on blended rates
Context window
The maximum number of tokens a model can consider at once, counting the prompt and the response together.
Filling one is a cost event rather than a capability question: a large window is only useful if you can afford to fill it.
Long-context surcharge
A higher rate applied once a prompt crosses a threshold, often 200,000 tokens.
Several providers re-bill the entire request at the higher rate rather than charging only for the excess, so crossing the line can double the cost of a call rather than adding to it incrementally.
List price
The standard public rate a provider posts for its own API. Every figure on this site is one.
Excludes enterprise agreements, committed-use discounts and reseller markups, none of which are public or comparable across providers.
Tokenizer
The component that splits text into tokens. Different model families use different ones.
This is why a model switch changes your token counts slightly even when the text is identical, and why a token estimate from one provider's tool is only approximate for another's.
Embeddings
Numeric representations of text used for search and retrieval. Priced per million tokens like generation, but usually far cheaper, and with no output side to pay for.
Retrieval-augmented generation (RAG)
Fetching relevant documents and putting them in the prompt so the model can answer from them.
Cost-wise it is an input-heavy pattern: retrieved context is billed as input tokens on every single call, which is what makes caching and input rates matter more than output rates.
Fine-tuning
Training a base model further on your own data. Billed as a one-off cost per million training tokens, plus usually a higher inference rate afterwards.
Cost per request
One API call priced end to end. Usually a fraction of a cent, which is exactly why it is worth knowing before multiplying by volume.