# The three matchers What exact, normalized and semantic each match on, what each costs, how each fails, and how to tell which one served a hit. Source: https://tokentriage.com/docs/build/caching-strategies --- import Callout from '../../../components/Callout.astro'; import NextSteps from '../../../components/NextSteps.astro'; A rule's `strategy` names one matcher or an ordered chain of them. Each matcher answers one question, "is there a stored entry for this request?", and answers it differently. The chain is walked in order and the first matcher that returns a candidate wins; later matchers are not consulted. Chaining is how you get a cheap fast path in front of an expensive one. `[exact, normalized, semantic]` costs a hash probe on the common case and only reaches the embedding call when both deterministic tiers have missed. ```yaml rules: - id: chat-hybrid match: { endpoint: chat } mode: shadow strategy: [exact, normalized, semantic] ``` Each matcher stores under its own partition, so the same request writes one entry per matcher in the chain. A three-matcher rule writes three entries per miss. ## `exact` Matches a request whose meaning-bearing content is identical to a stored one after canonical encoding. ```yaml strategy: exact ``` **How it decides.** It builds the request's semantic surface (the message array, the system prompt, the tool schema, and the sampling-parameter class), encodes it as canonical JSON, and hashes that together with the dialect, endpoint kind, served provider, served model and stream flag. Two requests that differ only in whitespace between JSON tokens, key order, number formatting or Unicode escaping produce the same bytes and therefore the same key. Any difference in content or in a keyed parameter produces a different one. **Knobs.** None. The matcher takes no parameters and rejects any it is given. **What it costs.** One hash and one store probe. Key derivation over a four-turn body benchmarks at about 9.9 µs, and an in-memory probe at about 0.25 µs, on the reference machine in [benchmarks](/docs/reference/benchmarks). **How it fails.** It misses. There is no partial match and no fallback inside the matcher; the chain moves on. **How to tell it is working.** `x-tt-cache-kind: exact` on the response, `"strategy": "exact"` on the row, and an `exact` entry in `hits_by_kind` on `/tt/api/v1/cache/stats`. **Reach for something else when** your callers send the same question with cosmetic text differences such as trailing spaces, a doubled space or a newline, because those are content differences to this matcher. ## `normalized` Matches after a declared text transform, so cosmetic whitespace differences in the *content* stop fragmenting the cache. ```yaml strategy: [exact, normalized] ``` **How it decides.** It runs the exact profile's canonicalization, then applies one transform to every string value in the document: runs of whitespace collapse to a single space, and leading and trailing whitespace is stripped. It hashes the result under a different profile id, so a normalized entry and an exact entry are separate entries even when their material coincides. It applies nothing that changes meaning. No stopword removal, no punctuation stripping, no stemming. Those belong to the semantic tier, where they are disclosed as a similarity score rather than presented as an identity. **Knobs.** A case-folding option exists in the matcher and has no configuration key today: the config-to-engine mapping does not carry per-matcher parameters, so the matcher always compiles with case folding off. Normalized matching is whitespace-only, and `Reset my password` and `reset my password` are different requests. **What it costs.** One extra string pass over the canonical document, then the same hash and probe as `exact`. It is scheduled in the same class as `exact`: no embedding, no deadline. **How it fails.** It misses. **How to tell it is working.** `x-tt-cache-kind: normalized`. In a chain behind `exact` this appears only for requests whose exact key missed, which is what you want to see. **Reach for something else when** the variation you are trying to absorb is a rewording rather than whitespace. Two spellings of the same question are not the same string after normalization. ## `semantic` Matches a request whose recent user text is close in embedding space to a stored one, above a threshold you set. ```yaml embedders: potion: { kind: static } rules: - id: chat-hybrid match: { endpoint: chat } mode: shadow strategy: [exact, normalized, semantic] ttl: 15m semantic: embedder: potion # names a cache.embedders entry threshold: 0.95 # at or above ⇒ served without a verifier low_threshold: 0.85 # below ⇒ miss; between ⇒ the verifier decides verifier: kreciprocal # none | kreciprocal | judge history_max_messages: 3 # embed the last N user turns; also an eligibility bound ``` **How it decides.** It embeds the last `history_max_messages` user turns, joined in order, and nothing else. Everything else the request contains (the system prompt, the tool schema, the sampling parameters, the assistant turns, and any earlier user turns outside the window) is hashed into the partition the search runs against, so a candidate stored under different context is not reachable at all. Within that partition it recalls the ten nearest stored vectors, recomputes the exact cosine against each candidate's stored vector, keeps the best, and applies the dual-threshold decision. **Knobs.** | Key | Default | What it does | |---|---|---| | `embedder` | — | required; names a `cache.embedders` entry | | `threshold` | — | the bar a score clears to serve with no verifier; must be in (0,1] | | `low_threshold` | `0` | the floor below which a score is a miss; `0` means no gray band exists | | `verifier` | `""` | the gray-band gate; without a synchronous one the gray band cannot serve | | `history_max_messages` | `0` | user turns embedded, and the history bound above which the request is not semantically served; `0` means no bound | | `verified` | absent | the calibration policy that decides whether these hits may report a dollar | **What it costs.** One embedding call per lookup that reaches this matcher, on the request path. With an in-process embedder that is CPU time and no money. With an HTTP embedder it is a network round trip and, on the external tier, a real charge that lands on the row as `embed_cost`. Recall at a ten-thousand-entry working set benchmarks at about 2.9 ms with the default exact index at 384 dimensions. **How it fails.** Toward not serving. A score below the floor is a miss; a gray-band candidate with no verifier configured is a miss; a verifier that errors is a miss; an embedding error is a fault that forwards live; a lookup that exceeds the semantic budget is a counted miss with `bypass_reason: over_budget` on the row. It can also fail by serving the wrong answer, which is what [the correctness page](/docs/build/caching-semantic) is about. **How to tell it is working.** `x-tt-cache-kind: semantic` with `x-tt-cache-similarity`, `-threshold` and `-verified` beside it, and `similarity` / `threshold` / `zone` on the row. **Reach for something else when** your traffic is machine-generated and already identical, when answers are personalised or time-sensitive, or when you cannot yet say what wrong-hit rate you are willing to accept. A semantic rule is two opt-ins: the strategy must list `semantic` **and** name a configured embedder. Validation refuses the first without the second, and warns on every enabled semantic rule that a hit may be a wrong answer to a similar-but-different request. ## Choosing a chain | Traffic | Chain | |---|---| | identical repeats from a service or an agent loop | `exact` | | human-typed repeats with cosmetic whitespace variation | `[exact, normalized]` | | the same question in different words, and you can measure wrong hits | `[exact, normalized, semantic]` | Start at the top of that table. The first two tiers cost microseconds, call nothing, and cannot serve an answer to a different question.