# Why didn't this request hit?
Work back from an unexpected cache miss to its cause, using the response header, the ledger row, and one key segment at a time.
Source: https://tokentriage.com/docs/build/caching-misses
---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
You sent what you are sure is a repeat and got `x-tt-cache: miss`. This page starts from what you
have in front of you — a response header and a request id — and works back.
Most of the time the answer is that the request was not a repeat in the sense the cache means. The
rest of the time the cache never looked.
## Start with the header, then the row
The header tells you which of two questions to ask.
| Header | It means | Ask |
|---|---|---|
| `x-tt-cache: bypass` | the cache did not look | why it declined |
| `x-tt-cache: miss` | the cache looked and found nothing servable | which key segment differed |
| `x-tt-cache: error` | a fault degraded to a live call | what faulted |
| no `x-tt-cache` header at all | the cache is not enabled on this deployment | `cache.enabled` |
Then pull the row for that request id and read `decision.response_cache`. One field carries the
cause on a bypass:
```json
"response_cache": {
"outcome": "bypass",
"rule": "chat-cache",
"bypass_reason": "tool_calls",
"stored": false
}
```
A `would_hit` row pairs with an `x-tt-cache: miss` header, and both are correct. The header
describes what the client received, and under a shadow rule the client received a live forward. If
you are counting hits, count rows, not headers.
## The bypass reasons
The vocabulary is closed. Each value names one gate, in the order they are evaluated.
| `bypass_reason` | What happened | What to change |
|---|---|---|
| `disabled` | `cache.enabled` is false | enable the block |
| `no_rule` | no rule's `match.endpoint` matched | add a rule, or drop `match.endpoint` from one |
| `client_off` | the request carried `x-tt-cache: off` | the caller's header |
| `non_cacheable_endpoint` | not `chat` or `responses` | nothing; see [the limits](/docs/build/caching-limits) |
| `tool_calls` | the request declares tools or carries tool results | nothing; tool traffic is not cached in v1 |
| `structured_output` | a semantic-only rule met `response_format` or a forced `tool_choice` | add `exact` to the chain, which can serve these |
| `history_too_long` | a semantic-only rule met more user turns than `history_max_messages` | raise the bound, or add a deterministic tier |
| `over_budget` | the semantic lookup passed its deadline | see [budgets](/docs/build/caching-performance) |
Two of these are commonly misread. `tool_calls` applies to the whole cache, not only the semantic
tier: a request that declares a tool schema is not cached at all, by any matcher.
`structured_output` and `history_too_long` are semantic-tier gates, and in a chain they skip the
semantic link rather than bypassing the request — you only see them as a `bypass_reason` when
`semantic` is the rule's only matcher.
`over_budget` arrives on a row whose `outcome` is `miss`, not `bypass`. A lookup that ran out of
time is counted as a miss that fell back to a live call.
## A miss you did not expect: walk the key
If the row says `miss`, the lookup ran and no stored entry matched. Something in the key differed.
Work down this list; it is ordered by how often each one is the answer.
1. **Is anything stored yet?** A first request stores; only the second can hit. Check the earlier
row for `"stored": true`.
2. **Did the route change?** The key names the *served* provider and model. Compare `decision.tier`,
`decision.upstream` and `decision.model_rewrite` on the two rows. A tier whose model changed
between them is a new key.
3. **Did a sampling parameter change?** `temperature`, `top_p`, `max_tokens`, `n`, `seed`,
`response_format`, `tool_choice`, `reasoning_effort` and the penalties are all key material, and
so is `stream_options.include_usage`. A client that starts sending a value it used to omit has a
new key.
4. **Is one streamed and the other not?** The stream flag is a key segment.
5. **Is it the same scope?** Compare `scope_sha7` on the two rows. Different values mean the two
requests were looked up in different partitions, usually because a dimension header differed.
6. **Has it expired?** Compare the gap between the rows against the rule's `ttl`. An entry past its
deadline is a miss on read.
7. **Did the process restart?** With no `l2.path` configured there is no durable tier, and the
store starts empty.
8. **Is the text only nearly identical?** Whitespace differences need `normalized` in the chain, and
rewordings need `semantic`.
From a sandbox running one rule, `strategy: [exact, normalized]`, in shadow. The same question was
sent five ways. What the ledger recorded:
```text
outcome kind strategy stored bypass_reason serving_tier
miss - - true - -
would_hit exact exact false - l1
would_hit exact exact false - l1
would_hit normalized normalized false - l2
bypass - - false tool_calls -
bypass - - false client_off -
miss - - true - -
miss - - true - -
```
Reading down: the first request stored. The next two were byte-identical and matched on `exact`. The
fourth added two extra spaces inside the question, so the exact key differed and `normalized`
caught it. The fifth declared a tool schema and was bypassed before any matcher ran. The sixth sent
`x-tt-cache: off`. The last two are the interesting ones: one carried a different tenant header, so
its scope differed, and one added `temperature: 0.2`, so its fingerprint differed. Neither could
match, and neither is a defect.
The `serving_tier: l2` on the normalized match is also real: the process had restarted between the
store and the lookup, so the in-memory tier was cold and the durable tier answered.
## Traps that make the answer non-obvious
**The preview command over-counts.** `tokentriage cache preview` groups rows by the ledger's
messages-only prompt hash plus model, endpoint and a three-field parameter class. That is a coarser
key than the cache's, so its repeat count is an upper bound. On the sandbox above it reported eight
repeat rows and an 80.0% repeat rate where the cache produced two hits and three would-hits.
**A hit row still names a tier and an upstream.** `decision.tier` and `decision.upstream` are stamped
by routing, which runs before the cache lookup. On a hit they describe the route the request would
have taken. No upstream call happened.
**`match.model`, `match.tier`, `match.dialect` and `match.dims` do not narrow a rule.** They
validate and then have no effect, so a rule you wrote to apply to one model is applying to
everything. See [cache rules](/docs/build/caching-rules#what-match-can-and-cannot-narrow).
**Governance runs before the cache.** A request can be turned away by a spend or rate limit before it
ever reaches a lookup that would have served it for free. Under token-per-minute pressure a
replay-eligible request takes a reservation, which is reconciled to zero once the replay finishes
with no tokens, but the reservation was momentarily held. That ordering is intended: a warm cache
must not serve where a live call would have been refused.
**A semantic score below the threshold can still serve.** If the score lands in the gray band and a
synchronous verifier passes, the hit is served with `x-tt-cache-verified: true` and a similarity
below `threshold`. That is the band working, not a bug.
## What to change
| Symptom | Cause | Fix |
|---|---|---|
| everything misses, no `x-tt-cache` header | cache not enabled | add `cache.enabled: true` |
| hits vanish after a deploy | the served model or upstream changed | expect a cold period; nothing to fix |
| hits vanish after every restart | no durable tier | set `cache.store.l2.path` |
| whitespace variants each miss | exact-only chain | add `normalized` to `strategy` |
| rewordings each miss | no semantic tier | read [the correctness page](/docs/build/caching-semantic) first |
| one tenant's traffic never shares | working as configured | `cache.scope` |
| all tenants share | `ledger.dimensions.enabled` is false | enable dimensions |
| `over_budget` on many rows | semantic lookup too slow for the corpus | raise `lookup_semantic_ms` or switch the index |