# How a request becomes a cache key
The seven fingerprint segments and the scope prefix that decide whether two requests can share a stored answer.
Source: https://tokentriage.com/docs/build/caching-keys
---
import CacheKeyDiagram from '../../../components/CacheKeyDiagram.astro';
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
The most common question anyone asks about a cache is why a request they thought was a repeat
didn't hit. Almost every answer is the same shape: something the key includes was different, and
you weren't thinking of it as part of the request.
So it is worth knowing what the key is made of. There are two parts, and they answer two different
questions. The **fingerprint** answers "would this request produce the same response?" The
**scope** answers "who is allowed to share that response?" An entry's identity is the hash of both,
and a lookup that has one without the other cannot read anything.
Read it left to right: the request splits into the seven fingerprint segments on top and the scope
segments below, each side is hashed, and the two hashes are hashed together into the entry id that
storage is keyed on.
## The fingerprint: would this produce the same answer?
Seven segments, joined with a length prefix on each so no two different splits can produce the same
byte stream, then hashed with SHA-256.
| Segment | Why it is in the key |
|---|---|
| dialect id | the same conversation in the OpenAI and Anthropic dialects produces different client bytes |
| endpoint kind | `chat` and `responses` return different shapes |
| **served provider** | the provider that actually answered, not the one the client asked for |
| **served model** | the model after any routing rewrite, not the requested name |
| profile id | `exact` or `normalized`; the same content under two profiles is two entries |
| canonical material | the meaning-bearing content, below |
| stream flag | a streamed entry cannot answer a non-streamed request |
The two bolded segments are the ones that surprise people. The lookup runs *after* routing, so the
key names the upstream and model that would serve this request now. Change a tier's model, route a
request to a different upstream, or add a model rewrite, and every entry stored under the old
identity becomes unreachable. That is the correct behaviour, because a stored Haiku answer is not a Sonnet
answer, and it means a routing change silently empties the effective cache.
### The canonical material
The material is not the raw request body. It is the response-determining content, re-encoded
canonically so that formatting cannot fragment the cache:
- the message array, verbatim per message and key-sorted
- the system prompt, when present
- the tool schema, when present
- the sampling-parameter class
Whitespace between JSON tokens, key order, number formatting and Unicode escaping all disappear
into the canonical encoding. Two clients that serialize the same conversation differently share an
entry.
The parameter class is the part worth reading twice, because each of these makes a new entry:
`temperature` · `top_p` · `max_tokens` · `n` · `frequency_penalty` · `presence_penalty` ·
`response_format` · `tool_choice` · `seed` · `stream_options.include_usage` · `reasoning_effort` ·
`size` · `quality`
An absent parameter and a parameter set to its default value are not the same thing. A client that
starts sending `temperature: 1.0` explicitly stops matching entries written before it did.
The parsed request carries derived fields that are not key material. Counts and sizes
(`messages`, `prompt_bytes`, `system_bytes`, `tool_count`) are excluded because the content they
summarize is already in the canonical material, and including a count as well would only add a way
to disagree with it. The messages-only prompt hash is excluded for the same reason: the canonical
material supersedes it, and it covers less.
One exclusion is a decision rather than a redundancy: whether the request carries provider
prompt-cache markers. Those affect the provider's billing for a call, not the content of the
response, so two requests differing only in a `cache_control` breakpoint share an entry.
The exclusion list is enforced at compile time. Adding a field to the parsed-request type breaks the
build in the key derivation until someone decides whether the new field belongs in the key.
## The scope: who may share it
The scope key is a mandatory prefix of every storage key and every index partition. An entry written
under one scope is not reachable from a probe under another: not filtered out afterwards, but
unable to be addressed. It is hashed from four things:
| Segment | What it does |
|---|---|
| cache generation epoch | lets a whole generation of entries be abandoned at once |
| matcher id | `exact`, `normalized` and `semantic` each own their entries |
| embedder fingerprint | empty for non-semantic matchers; changing the embedder repartitions |
| the rule's ordered scope dims | `tenant=acme`, `user=…`, and so on |
The matcher id in the scope is why a three-matcher chain writes three entries per miss, and why
promoting a rule from `[exact]` to `[exact, normalized]` starts cold for the new tier.
Which dimensions appear, and what changing them does to isolation and to hit rate, is
[cache scope](/docs/build/caching-scope).
## The semantic key is a different shape
Semantic matching has no fingerprint to compare, because the point is that the text differs. It
partitions instead. The scope key is folded together with the *context material*, meaning everything
response-determining that the embedded text leaves out, into a partition. Recall runs only inside
it.
Concretely, for a rule with `history_max_messages: 3`, the embedded text is the last three user
turns joined with newlines. The system prompt, the tool schema, the parameter class, the assistant
turns and any earlier user turns are context material. Two requests can only be compared by
similarity when all of that matches byte for byte after canonical encoding.
Similarity is applied to recent user text and to nothing else. Everything outside that window has
to match exactly for a candidate to be reachable at all, which is the main reason the wrong-hit
surface is smaller than "an embedding said these were close".
## Two entry ids, in practice
The row carries the first seven characters of each hash: `entry_sha7` and `scope_sha7`. The
response header carries the full entry id, which is also the handle a per-entry purge takes.
```json
"entry_sha7": "3a2e388",
"scope_sha7": "799cda7",
```
Two rows sharing a `scope_sha7` were looked up in the same partition. Two rows sharing an
`entry_sha7` matched the same stored answer. A raw tenant id never appears in either.
## What can go wrong
**A deploy empties the cache without deleting anything.** Changing a tier's model, adding a rewrite,
switching a rule's matcher chain, changing `cache.scope`, or pointing a semantic rule at a different
embedder all change the key. The old entries are still on disk and no probe can reach them.
**A client library upgrade stops the hits.** If it starts sending a parameter it used to omit, every
request has a new key. Compare two request bodies before you widen a TTL.