# What a cache saving means Why a hit row licenses a dollar, a would-hit does not, a semantic hit waits, and why this is never added to a provider discount. Source: https://tokentriage.com/docs/build/caching-savings --- import TwoCachesDiagram from '../../../components/TwoCachesDiagram.astro'; import Callout from '../../../components/Callout.astro'; import Details from '../../../components/Details.astro'; import NextSteps from '../../../components/NextSteps.astro'; A saving is a claim about money you did not spend, and money you did not spend leaves no receipt. Every such figure therefore rests on something, and the useful question about any of them is what. For the response cache the answer is unusually good. A hit is a call that did not go upstream, and the call it replaced actually happened once, with real token counts recorded. So the claim is: this request would have cost what the identical earlier request cost. That is still an estimate, and this page is about which parts of it are solid and which are not. ## How the figure is computed On a hit, the stored entry carries the source request's usage. That usage is repriced under the *current* pricing snapshot for the stored provider and model, through the same path every other dollar in the product travels. ```json "source_usage": { "input_tokens": 1200, "output_tokens": 350, "total_tokens": 1550 }, "saved": { "usd": 0.00039, "estimated": true, "unpriced": false }, "saved_basis": "source_usage repriced @ litellm@2026-07-14#179affb, fixed usage" ``` That row came from a sandbox against a mock upstream, so the tokens are the mock's and the dollar is arithmetic you can check: 1,200 × $0.15/M + 350 × $0.60/M. Three things about it: **It is always an estimate.** `estimated: true` is not a hedge about the arithmetic. It says the token counts are the *earlier* request's, held fixed. A fresh generation would have produced a different number of output tokens; nobody knows how many. **It uses today's prices, not the prices at store time.** The basis string names the snapshot, so a figure is re-derivable. **It is unpriced-safe.** If the stored provider and model do not resolve to a price, the field records `unpriced: true` and makes no dollar claim. It does not report `$0`. The count of such hits is disclosed separately as `unpriced_hits`. The hit row's own cost is zero, and that is a different statement: `usage.source` is `replay` with zero token buckets, so replayed tokens are never counted as fresh spend anywhere. ## Which rows may contribute Three populations, kept apart. | Population | Contributes to the licensed figure | Why | |---|---|---| | `hit` rows | yes | the call did not happen | | `would_hit` rows | no | the request forwarded and was paid for | | semantic `hit` rows without a calibrated policy | no, counted separately | nothing has measured the wrong-hit rate | The stats surface reports the first two under separate basis strings and does not add them: ```json "savings": { "usd": 0, "estimated": true, "basis": "response_cache hit rows, saved repriced from source usage (hit-only)", "verdict_ref": "/tt/api/v1/cache/evidence" }, "shadow_savings": { "usd": 0.00078, "estimated": true, "basis": "shadow-mode would-save, nothing avoided" } ``` The `$0` licensed figure above is from a rule still in shadow. It is a not-yet-measured zero, and `estimated: true` beside it says so. A shadow would-save is what the cache *would* have avoided; the money was spent. Never add `shadow_savings` to `savings`. The would-hit rows are rows you paid for. A projection and a measurement have different meanings and the difference between them is not a number. Both figures carry a `verdict_ref` pointing at the evidence surface, and a savings value that lacks its basis or that reference cannot be serialised at all — the API refuses to emit an unlabelled cache dollar. ## The semantic exclusion A semantic hit's saving appears on its row and does not appear in the licensed total. ```json "savings": { "usd": 0, "excluded_semantic": 1, "basis": "…(hit-only)" } ``` The `excluded_semantic` count is the disclosure: one hit occurred, its dollar was withheld. The evidence surface says why — either no calibration policy is configured, or one is and has not accumulated enough labels to bound the error rate. Either way it reads `cannot_measure` with a reason rather than producing a smaller confident number. Once a calibrated policy licenses the kind, the evidence row switches to `licensed` and reports the effective threshold, the lower bound on the correct-serve rate, and the sample count behind it, so the decision can be re-derived. [When a semantic hit is wrong](/docs/build/caching-semantic) covers the calibration itself. ## The two caches are not summed The diagram separates them: on the left the provider's prompt cache discounting a call that happened, on the right this cache removing a call that would have happened. Both produce a dollar and the dollars mean different things. A prompt-cache discount is provider-reported, priced from tokens on your bill. A response-cache saving is a counterfactual over a request that produced no bill at all. Adding them charges one request to two ledgers. [Cache accounting](/docs/concepts/cache-accounting) is the canonical statement of that rule, with the arithmetic worked through. Nothing on this page or anywhere in this section adds the two.
Semantic caching's embeddings are spend, and a metered embedder's charge lands on the row as `embed_cost` and joins the metacost surface. On a rule whose semantic tier is not yet licensed to report savings, that is spend against a saving of zero. The measurement's own cost is [Cost & attribution](/docs/build/cost-metacost)'s subject; this section owns where the number is produced.
## Where this figure appears elsewhere The cache stats and evidence surfaces are the authority. Downstream, the same population feeds the savings verdict vocabulary as `cache-licensed-replay-v1`, which is the method name a cost surface prints beside a claimed cache dollar. [Verdicts, methods and baselines](/docs/build/cost-figure-reference) is where that vocabulary is defined; the reason a cache figure earns its strongest class is the replay evidence described on this page. ## What can go wrong **A finance-facing total silently excludes rows.** Quote the exclusions with the number: unpriced hits and excluded semantic hits are both reported and both mean the total is smaller than the row count implies. **A big shadow figure gets forwarded as a saving.** It is labelled, and labels get lost in screenshots. Say "would have saved" out loud when you quote it. **The figure changes without traffic changing.** The reprice runs under the current snapshot, so a pricing update moves it. That is intended; the basis string names the snapshot each figure used.