Why didn't this request hit?

You sent what you are sure is a repeat and got x-tt-cache: miss. This page starts from what you have in front of you — a response header and a request id — and works back.

Most of the time the answer is that the request was not a repeat in the sense the cache means. The rest of the time the cache never looked.

Start with the header, then the row

The header tells you which of two questions to ask.

Then pull the row for that request id and read decision.response_cache. One field carries the cause on a bypass:

"response_cache": {
  "outcome": "bypass",
  "rule": "chat-cache",
  "bypass_reason": "tool_calls",
  "stored": false
}

The bypass reasons

The vocabulary is closed. Each value names one gate, in the order they are evaluated.

Two of these are commonly misread. tool_calls applies to the whole cache, not only the semantic tier: a request that declares a tool schema is not cached at all, by any matcher. structured_output and history_too_long are semantic-tier gates, and in a chain they skip the semantic link rather than bypassing the request — you only see them as a bypass_reason when semantic is the rule’s only matcher.

over_budget arrives on a row whose outcome is miss, not bypass. A lookup that ran out of time is counted as a miss that fell back to a live call.

A miss you did not expect: walk the key

If the row says miss, the lookup ran and no stored entry matched. Something in the key differed. Work down this list; it is ordered by how often each one is the answer.

1. Is anything stored yet? A first request stores; only the second can hit. Check the earlier row for "stored": true.
2. Did the route change? The key names the served provider and model. Compare decision.tier, decision.upstream and decision.model_rewrite on the two rows. A tier whose model changed between them is a new key.
3. Did a sampling parameter change? temperature, top_p, max_tokens, n, seed, response_format, tool_choice, reasoning_effort and the penalties are all key material, and so is stream_options.include_usage. A client that starts sending a value it used to omit has a new key.
4. Is one streamed and the other not? The stream flag is a key segment.
5. Is it the same scope? Compare scope_sha7 on the two rows. Different values mean the two requests were looked up in different partitions, usually because a dimension header differed.
6. Has it expired? Compare the gap between the rows against the rule’s ttl. An entry past its deadline is a miss on read.
7. Did the process restart? With no l2.path configured there is no durable tier, and the store starts empty.
8. Is the text only nearly identical? Whitespace differences need normalized in the chain, and rewordings need semantic.

outcome        kind         strategy     stored  bypass_reason  serving_tier
miss           -            -            true    -              -
would_hit      exact        exact        false   -              l1
would_hit      exact        exact        false   -              l1
would_hit      normalized   normalized   false   -              l2
bypass         -            -            false   tool_calls     -
bypass         -            -            false   client_off     -
miss           -            -            true    -              -
miss           -            -            true    -              -

Traps that make the answer non-obvious

The preview command over-counts. tokentriage cache preview groups rows by the ledger’s messages-only prompt hash plus model, endpoint and a three-field parameter class. That is a coarser key than the cache’s, so its repeat count is an upper bound. On the sandbox above it reported eight repeat rows and an 80.0% repeat rate where the cache produced two hits and three would-hits.

A hit row still names a tier and an upstream. decision.tier and decision.upstream are stamped by routing, which runs before the cache lookup. On a hit they describe the route the request would have taken. No upstream call happened.

match.model, match.tier, match.dialect and match.dims do not narrow a rule. They validate and then have no effect, so a rule you wrote to apply to one model is applying to everything. See cache rules.

Governance runs before the cache. A request can be turned away by a spend or rate limit before it ever reaches a lookup that would have served it for free. Under token-per-minute pressure a replay-eligible request takes a reservation, which is reconciled to zero once the replay finishes with no tokens, but the reservation was momentarily held. That ordering is intended: a warm cache must not serve where a live call would have been refused.

A semantic score below the threshold can still serve. If the score lands in the gray band and a synchronous verifier passes, the hit is served with x-tt-cache-verified: true and a similarity below threshold. That is the band working, not a bug.

What to change