Two kinds of cache savings

Two dashboards can report “cache savings” for the same month without measuring the same thing. One counts discounts on requests sent to a model provider. The other estimates the cost of requests that TokenTriage did not send.

Keep the figures separate unless your own analysis gives them the same time window, set of requests and comparison baseline. This page shows where each number comes from and how to read it.

If tokens and per-token rates are new to you, read tokens, prices, and where the money goes first. If you want to configure TokenTriage’s cache, go to response caching.

The two caches

The caches sit at different layers of a request. The diagram shows whether the model provider receives a request at all.

Read the diagram from left to right. A provider prompt-cache hit still reaches the provider, but some input tokens cost less. A TokenTriage response-cache hit stops before the provider and returns a stored response.

Question	Provider prompt cache	TokenTriage response cache
Who owns it?	the model provider	TokenTriage
What is reused?	work on a repeated prompt prefix	a stored response
Does the provider receive a request?	yes	no
Financial effect	some input tokens cost less	the provider call is avoided

Provider prompt cache. A prompt prefix is the stable beginning of a request, such as a system message, a tool schema or a document attached to many calls. When the prefix repeats, the provider may reuse work it has already done. The provider still processes the request and returns usage, but charges a lower rate for the cached input tokens.

TokenTriage response cache. TokenTriage stores a provider response and can return it when a later request matches. A hit makes no provider call, so there are no new provider tokens to bill. TokenTriage estimates the avoided cost from the usage recorded with the stored response.

Why TokenTriage reports them separately

Two dollar figures are safe to combine only when they cover the same period and use a compatible baseline. A baseline is the cost you are comparing against. These two figures begin with different request populations and different comparisons:

Figure	Which rows contribute?	What is compared?	Evidence
provider prompt-cache saving	requests that reached the provider	fresh-input price versus the provider’s cached-input price	cache-token counts reported by the provider
TokenTriage response-cache saving	response-cache hit rows	the stored source usage repriced at current rates versus no provider spend on the hit	historical source usage and the current price table

Consider two consecutive requests. Request A reaches the provider and reports cached input tokens; any prompt-cache discount belongs to that billed row. Request B is a TokenTriage response-cache hit, so it has no provider usage of its own. Its estimated saving comes from Request A’s stored usage.

That stored usage can itself contain prompt-cache reads or writes. TokenTriage keeps those token categories when it reprices the source request. As a result, the estimate answers “what would a request like the stored one cost now?” It does not construct a second, cache-free version of the request.

You can build a combined analysis, but you must first define a shared no-cache baseline and decide how each request would have behaved without either cache. TokenTriage does not invent that additional counterfactual. Its dashboard keeps the two supported figures side by side.

Where each one appears

The field names identify which cache produced a figure.

Provider prompt cache. The provider’s usage becomes separate usage.cache_read_tokens and usage.cache_write_tokens categories. Cache reads are tokens served from the provider’s cache. Cache writes are tokens used to fill that cache. TokenTriage prices both categories and records them in the decision ledger, a newline-delimited JSON (JSONL) file with one row per proxied request. See decision ledger fields for the complete row shape.

TokenTriage response cache. Its outcome appears in the response_cache block on the ledger row. On a hit, that block also carries the estimated saving. The response includes x-tt-cache-* headers describing the outcome, match type, age and source (response headers). The row’s top-level token counts are zero because no new provider work happened. The original usage remains inside the response_cache block as the basis for the estimate.

The dashboard uses two short labels:

Label	Plain-language meaning
provider-reported, priced	provider usage was received and TokenTriage applied the matching token rates
counterfactual, licensed	the call did not happen; TokenTriage has enough evidence to include its estimated avoided cost

If the stored provider or model has no resolvable price, TokenTriage leaves the response-cache saving blank instead of reporting $0.00. The detailed rules for which hit rows qualify are in what a cache saving means.

Shadow-mode “would-hit” rows stay separate too. In shadow mode, TokenTriage records what the cache would have served but still forwards and pays for the request. Because no spend was avoided, those projections do not contribute to the response-cache savings total.

Prompt-cache writes have a break-even

Prompt caching may cost more before it saves money. Filling an entry can carry a write premium, while later reads receive the discount. The break-even is the number of cached reads needed to repay that premium:

write premium per token = cache-write rate - input rate
saving per cached read  = input rate - cache-read rate
break-even reads        = round up(write premium / saving per read)

Round up because a fraction of a request cannot repay the premium.

For example, the price table shipped with this build lists these rates for anthropic/claude-sonnet-4-5: $3.00 per million input tokens, $3.75 per million cache-write tokens and $0.30 per million cache-read tokens. The write premium is $0.75 per million tokens. Each cached read saves $2.70 per million, so one read repays the premium.

Here is the same example for a 20,000-token prefix used ten times:

without the prompt cache
  10 requests x 20,000 tokens @ $3.00/M          = $0.600

with the prompt cache
  1 write      20,000 tokens @ $3.75/M           = $0.075
  9 reads     180,000 tokens @ $0.30/M           = $0.054
                                                   = $0.129

prompt-prefix saving                              = $0.471

This arithmetic covers only the repeated prefix. The rest of each request and all output tokens are billed normally. It also assumes all ten requests arrive before the cache entry expires. If the entry expires between requests, each new write pays the premium again.

A cache entry’s time to live (TTL) is how long the provider keeps it. Some providers charge different write rates for different TTLs. TokenTriage preserves a provider-reported 5-minute versus 1-hour split and prices each bucket at its own rate.

Run the calculation with the rates for your provider and model before planning around it.

When TokenTriage lacks a cache rate

TokenTriage can calculate the break-even only when its built-in price table, or your pricing override, supplies the input, cache-read and cache-write rates.

If a provider reports cache tokens but the table lacks the matching cache rate, TokenTriage uses the ordinary input rate for that token category and marks the cost as estimated. This avoids presenting an unknown discount as certain, but it also means the built-in data cannot show a useful break-even for that model.

A missing rate describes the price data in this build, not necessarily the provider’s current pricing. Override a price shows how to supply your own rate. Pricing keys explains how TokenTriage chooses the row to override.

This page does not cover how to enable either cache or decide which one fits your traffic. Provider prompt caching helps when a large, stable prefix repeats. TokenTriage’s response cache helps when whole requests repeat closely enough to reuse the answer. You can run both at once and account for them separately.