The three matchers

A rule’s strategy names one matcher or an ordered chain of them. Each matcher answers one question, “is there a stored entry for this request?”, and answers it differently. The chain is walked in order and the first matcher that returns a candidate wins; later matchers are not consulted.

Chaining is how you get a cheap fast path in front of an expensive one. [exact, normalized, semantic] costs a hash probe on the common case and only reaches the embedding call when both deterministic tiers have missed.

rules:
  - id: chat-hybrid
    match: { endpoint: chat }
    mode: shadow
    strategy: [exact, normalized, semantic]

Each matcher stores under its own partition, so the same request writes one entry per matcher in the chain. A three-matcher rule writes three entries per miss.

exact

Matches a request whose meaning-bearing content is identical to a stored one after canonical encoding.

strategy: exact

How it decides. It builds the request’s semantic surface (the message array, the system prompt, the tool schema, and the sampling-parameter class), encodes it as canonical JSON, and hashes that together with the dialect, endpoint kind, served provider, served model and stream flag. Two requests that differ only in whitespace between JSON tokens, key order, number formatting or Unicode escaping produce the same bytes and therefore the same key. Any difference in content or in a keyed parameter produces a different one.

Knobs. None. The matcher takes no parameters and rejects any it is given.

What it costs. One hash and one store probe. Key derivation over a four-turn body benchmarks at about 9.9 µs, and an in-memory probe at about 0.25 µs, on the reference machine in benchmarks.

How it fails. It misses. There is no partial match and no fallback inside the matcher; the chain moves on.

How to tell it is working. x-tt-cache-kind: exact on the response, "strategy": "exact" on the row, and an exact entry in hits_by_kind on /tt/api/v1/cache/stats.

Reach for something else when your callers send the same question with cosmetic text differences such as trailing spaces, a doubled space or a newline, because those are content differences to this matcher.

normalized

Matches after a declared text transform, so cosmetic whitespace differences in the content stop fragmenting the cache.

strategy: [exact, normalized]

How it decides. It runs the exact profile’s canonicalization, then applies one transform to every string value in the document: runs of whitespace collapse to a single space, and leading and trailing whitespace is stripped. It hashes the result under a different profile id, so a normalized entry and an exact entry are separate entries even when their material coincides.

It applies nothing that changes meaning. No stopword removal, no punctuation stripping, no stemming. Those belong to the semantic tier, where they are disclosed as a similarity score rather than presented as an identity.

Knobs. A case-folding option exists in the matcher and has no configuration key today: the config-to-engine mapping does not carry per-matcher parameters, so the matcher always compiles with case folding off. Normalized matching is whitespace-only, and Reset my password and reset my password are different requests.

What it costs. One extra string pass over the canonical document, then the same hash and probe as exact. It is scheduled in the same class as exact: no embedding, no deadline.

How it fails. It misses.

How to tell it is working. x-tt-cache-kind: normalized. In a chain behind exact this appears only for requests whose exact key missed, which is what you want to see.

Reach for something else when the variation you are trying to absorb is a rewording rather than whitespace. Two spellings of the same question are not the same string after normalization.

semantic

Matches a request whose recent user text is close in embedding space to a stored one, above a threshold you set.

embedders:
  potion: { kind: static }
rules:
  - id: chat-hybrid
    match: { endpoint: chat }
    mode: shadow
    strategy: [exact, normalized, semantic]
    ttl: 15m
    semantic:
      embedder: potion            # names a cache.embedders entry
      threshold: 0.95             # at or above ⇒ served without a verifier
      low_threshold: 0.85         # below ⇒ miss; between ⇒ the verifier decides
      verifier: kreciprocal       # none | kreciprocal | judge
      history_max_messages: 3     # embed the last N user turns; also an eligibility bound

How it decides. It embeds the last history_max_messages user turns, joined in order, and nothing else. Everything else the request contains (the system prompt, the tool schema, the sampling parameters, the assistant turns, and any earlier user turns outside the window) is hashed into the partition the search runs against, so a candidate stored under different context is not reachable at all. Within that partition it recalls the ten nearest stored vectors, recomputes the exact cosine against each candidate’s stored vector, keeps the best, and applies the dual-threshold decision.

Knobs.

Key	Default	What it does
embedder	—	required; names a cache.embedders entry
threshold	—	the bar a score clears to serve with no verifier; must be in (0,1]
low_threshold	0	the floor below which a score is a miss; 0 means no gray band exists
verifier	""	the gray-band gate; without a synchronous one the gray band cannot serve
history_max_messages	0	user turns embedded, and the history bound above which the request is not semantically served; 0 means no bound
verified	absent	the calibration policy that decides whether these hits may report a dollar

What it costs. One embedding call per lookup that reaches this matcher, on the request path. With an in-process embedder that is CPU time and no money. With an HTTP embedder it is a network round trip and, on the external tier, a real charge that lands on the row as embed_cost. Recall at a ten-thousand-entry working set benchmarks at about 2.9 ms with the default exact index at 384 dimensions.

How it fails. Toward not serving. A score below the floor is a miss; a gray-band candidate with no verifier configured is a miss; a verifier that errors is a miss; an embedding error is a fault that forwards live; a lookup that exceeds the semantic budget is a counted miss with bypass_reason: over_budget on the row. It can also fail by serving the wrong answer, which is what the correctness page is about.

How to tell it is working. x-tt-cache-kind: semantic with x-tt-cache-similarity, -threshold and -verified beside it, and similarity / threshold / zone on the row.

Reach for something else when your traffic is machine-generated and already identical, when answers are personalised or time-sensitive, or when you cannot yet say what wrong-hit rate you are willing to accept.

Choosing a chain

Traffic	Chain
identical repeats from a service or an agent loop	exact
human-typed repeats with cosmetic whitespace variation	[exact, normalized]
the same question in different words, and you can measure wrong hits	[exact, normalized, semantic]

Start at the top of that table. The first two tiers cost microseconds, call nothing, and cannot serve an answer to a different question.