Prompt caching and batch APIs: what they actually save

On a price list, prompt caching and batch processing look like enormous discounts. In our data, cached input is listed at 90% to 98% below the standard input price, and batch prices are 50% below standard prices. Few bills fall by anything like those percentages. The discount applies to a slice of your tokens, not to all of them, and the size of that slice depends on how your application is built.

This guide explains how each mechanism is billed, shows with worked examples why the real saving is smaller than the headline, and gives you a way to estimate your own figure before you rebuild anything. Prices come from each provider's published price list (sources on our methodology page). The traffic figures are illustrative.

How prompt caching is billed

When many requests start with the same long block of text (a system prompt, tool definitions, a reference document), the provider can keep the processed form of that opening block and reuse it. Later requests that begin with exactly the same tokens read that block from the cache and pay the cached input price for it. The rest of the prompt, and all of the output, is billed normally.

Three details decide how much you actually get back:

• Only a matching prefix counts. The cache works from the start of the prompt. If anything near the top changes between calls (a timestamp, the user's name, a reordered list of tools) everything after it is a miss.
• Only input tokens are discounted. Output is never cached. An output-heavy workload gains little.
• Writing and keeping a cache may cost extra. Depending on the provider, writing a block to the cache can cost more than a normal input token, and keeping it can carry an hourly storage fee. Our calculator and the examples below leave those charges out, so treat the savings shown as an upper bound and check the provider's page for the model you use.

Here are the published discounts in our data, comparing each model's own prices:

"n/a" means the provider does not publish that price, or we have not been able to verify it. We leave it blank rather than assume a figure.

Why the headline discount is not your saving

The saving from caching is the discount multiplied by the share of your bill it touches:

saving = cached share × input tokens × (input price − cached price) × requests saving as a share of the bill = saving ÷ total cost without caching

Take a support assistant on GPT-6.1 Terra, whose cached input price is 95% below its standard input price. It handles 10,000 requests a day with 6,000 input tokens (a long system prompt, policies and product documentation, then the conversation) and 400 output tokens per request.

Even with 80% of input tokens served from the cache, which takes deliberate prompt design, the bill falls by 57%, not 95%. The uncached part of the prompt and every output token are still billed at full price.

Now flip the shape. A drafting tool on the same model sends 800 input tokens and gets back 1,500 output tokens per request. With the same 80% of input cached, the monthly bill goes from $2,525 to $2,340, a saving of 7.3%. Caching is worth a lot to input-heavy applications such as retrieval, long system prompts and multi-turn chat, and very little to output-heavy ones.

What decides your cache share

• Prompt order. Put everything stable first (instructions, tool definitions, reference documents) and everything variable last (retrieved snippets that change per query, the conversation, the user message). Moving one dynamic line from the top of a prompt to the bottom can turn a near-total miss into a high hit rate.
• Traffic density. Caches expire. An endpoint that receives a request every few seconds keeps its cache warm; one that receives a request every hour may miss almost every time. Low-traffic tenants in a multi-tenant product often see far lower hit rates than the busiest ones.
• Provider rules. Providers differ in whether caching is automatic or must be requested, in the minimum length they will cache and in how long entries live. Read the caching section of the documentation for the specific model you use.

You do not have to guess the result. Most APIs report cached input tokens in the usage data of each response. Run a day of realistic traffic in staging, add up cached and uncached input, and use that ratio in the calculator.

How batch APIs are billed

A batch API takes a file of requests, processes them asynchronously and returns the results later, within a completion window the provider states in its documentation. In exchange for giving up an immediate answer, you pay less for both input and output. In our data, every model with published batch prices charges 50% less than its standard rate.

Because the batch discount applies to output as well as input, it does not have the shape problem that caching has. Moving the support assistant's full workload to batch would take it from $2,433 to $1,217 a month, a 50% saving. The catch is that a support assistant cannot use a batch API: its users are waiting.

Work that usually fits a batch job:

• Evaluation suites and regression tests run against new prompts or models.
• Backfills, such as classifying or extracting fields from an archive of documents.
• Nightly or weekly reports, digests and content moderation queues.
• Generating synthetic data or test cases.

Work that does not fit: anything a person is waiting on, and agent chains where each call needs the previous answer before it can start. You can batch each step of a chain, but the total time becomes the number of steps multiplied by the batch completion time.

Stacking caching and batch

Some providers publish a separate batch price for cached input, so the two discounts combine. For work that qualifies for both, the saving can be large: with 80% of input cached and every request batched, the same workload on GPT-6.1 Terra costs $523.17 a month, 78.5% less than the standard price.

Not every provider prints that combined figure. For Claude Fable 5.2, Claude Opus 5.5, Claude Sonnet 5.5 and Claude Haiku 5.5, there is no published batch cached-input price we could verify, so our calculator shows those models as unavailable for cached batch work rather than working out a figure the provider has not printed.

For Mistral Large 4 and Mistral Small 4, we have not yet verified published batch prices, so they can only be priced with the batch share set to zero.

A quick way to estimate your own saving

1. For each call type, measure the share of input tokens that is identical across calls and sits at the start of the prompt. That is your best possible cache share.
2. Multiply it by a realistic hit rate for your traffic. Busy, steady endpoints keep caches warm; sparse ones do not.
3. Separate the work that can wait hours from the work that cannot. Only the first can go to a batch API.
4. Enter the cache share and batch share in the calculator and compare the result with both set to zero. The comparison table shows the effect on every model at once.
5. Check the provider's page for cache write and storage fees on your model, and subtract them from the saving.

Done this way, the saving you plan for is the one you will see on the invoice. Both mechanisms are worth using wherever they fit. The point is to size them honestly, so a budget that assumes a 90% discount does not meet a bill that fell by a third.