What the cache will not do

Some requests are not cached, and some responses are not stored. Both lists are short and closed, and every case lands on the ledger row with a reason rather than passing silently.

Cacheable endpoints

Two, in v1: chat and responses. Everything else — embeddings, models, images, audio — is not cacheable, and naming one in a rule is a validation error:

cache.rules[0].match.endpoint: endpoint kind "embeddings" is not cacheable (chat|responses only in v1)

Embeddings responses are deterministic and repeat heavily, which makes them the obvious next candidate. The schema admits the endpoint and the tier is held back until the chat path has soaked.

The eligibility gates

Evaluated in order, after a rule has been selected and before any key is derived. Each produces a bypass_reason on the row.

Gate	Applies to	Reason recorded
the client sent x-tt-cache: off	the whole cache	client_off
the request could not be parsed	the whole cache	non_cacheable_endpoint
the endpoint is not cacheable	the whole cache	non_cacheable_endpoint
the request declares tools, or carries tool results	the whole cache	tool_calls
response_format, or a tool_choice that forces a call	the semantic matcher	structured_output
more user turns than history_max_messages	the semantic matcher	history_too_long

The first four turn the whole request away. The last two are semantic-only: in a chain the semantic link is skipped and the deterministic matchers still get their turn, because an exact match can serve a structured-output request perfectly well. You see them as a bypass_reason only when semantic is the rule’s sole matcher.

What is not stored

A miss writes the client-visible response to the store only when all of these hold:

• the status is 2xx
• the body is complete: not truncated by the body limit, and for a stream, a terminal finish reason with no error and no capture overflow
• the client did not send x-tt-cache: no-store
• the response was not served by a cascade
• the payload is within cache.store.l2.max_entry_kb

An aborted stream, a client that disconnected mid-response, an oversize body, and an error status are all left unstored. The row records "stored": false.

The size cap deserves a note. It is why max_entry_kb must be strictly below server.max_body_bytes: the observing buffer truncates at the body limit, so an entry cap at or above it would store truncated bodies as though they were complete. Validation refuses that config:

cache.store.l2.max_entry_kb: 10240 KB (10485760 bytes) must be strictly less than server.max_body_bytes (10485760 bytes)

What v1 does not cache

These are scoped out rather than missing, and each leaves a clean edge for later.

Binary event-stream responses. Server-sent-events streaming replays frame by frame and is fully supported. The AWS application/vnd.amazon.eventstream binary framing is not decoded for capture, so those responses are not stored.

Cascade-served responses. When a judge-graded cascade produces the answer, the response does not travel through the observation point the store rides, so nothing is written and the row records "stored": false. Reads are unaffected: a lookup still short-circuits a cascade, so the read side is complete and only the write side excludes them.

Endpoints other than chat and responses, as above.

Provider prompt-cache directive injection. TokenTriage forwards cache_control, cachePoint and cachedContent markers untouched, and prices the tokens they save. It does not place those markers for you: doing so mutates the client’s bytes and creates stateful objects with their own storage billing. See cache accounting.

Gemini explicit-cache storage fees. A time-by-size charge that appears in no response body, so it cannot be attributed to a request. It is named rather than approximated.

Request coalescing. Two identical requests in flight at once both go upstream. Single-flight would be the natural companion feature and it is not shipped; it is named here so it is not assumed.

Shared state across replicas, by default. L1 and L2 are per-process. Two replicas keep separate caches and a cold replica misses. The opt-in Redis tier shares payloads across replicas; the semantic recall index is still built per process. See the store.