Request and response capture

Every row in your ledger records a SHA-256 of the prompt and nothing else of its content. That is the shipped default: you get counts, costs and routing decisions, and the text of what anyone asked or was told stays out of your storage.

You can change that. Content capture stores request and response bodies so a record on the dashboard can show you what a request actually said. It is off until you turn it on, and this page is what you should read before you do.

What the default gives you

Without capture, a decision-log row holds prompt_sha256, a hash, alongside the metadata that makes the dashboard work: model, tier, tokens, cost, latency, the routing decision and any business dimensions you tagged. That is enough to attribute spend, compare tiers, explain a decision and reconcile an invoice.

What it is not enough for is debugging content. You cannot read the prompt that produced a bad answer, and you cannot tell two requests apart beyond their hashes matching or not.

What turning it on does

dashboard:
  capture:
    request: true          # default: false — stores prompt text
    response: true         # default: false — stores completion text
    ttl: 720h              # default: 720h (30 days); 0 = keep until retention sweeps
    path: /var/lib/tokentriage/content

The two sides are independent. Capturing requests stores the system prompt and messages; capturing responses stores the completion. Turn on one, the other, or both.

Bodies live in a sidecar store at capture.path, outside the append-only ledger. That separation is what makes them deletable: a captured body can be swept without touching the row it belongs to, so removing content leaves no hole in your cost history.

The dashboard states the consequence on the surface rather than in a tooltip:

localhost:9090/ui/settingsDemo data

Four properties to know before you enable it

Capture fails open. If writing a body fails for any reason, the request is unaffected, the ledger row is still written, and the row’s capture flag is false. Storing content cannot become a way for your gateway to break traffic.

Streaming responses are not capturable. A streamed completion is relayed byte-for-byte and never buffered, which is what makes the proxy transparent, and it also means there is no complete body to store. If your traffic is mostly streaming, response capture will get you much less than you expect. The surface discloses this.

Bodies are not inlined into the API. A record response carries a reference to captured content rather than the content itself. Reading the body is a separate, authenticated request against a content route that is proof against path traversal, and it requires the content:read scope — one of the seven privileged operations in roles and permissions.

The sweeper removes content only. An audited TTL sweep deletes captured bodies whose lifetime has elapsed. It leaves the ledger row alone, so a swept request keeps its cost, its decision and its hash, losing only its text.

Deciding whether to turn it on

Three questions, in this order.

Do you have a debugging problem capture would solve? The common one is a routing decision that looks wrong and cannot be explained from metadata. If your open questions are all about cost or attribution, capture will not answer them.

What is in your prompts? They are your users’ text. Turning capture on makes your TokenTriage volume part of whatever regime governs that text, and the store is a new place it lives. This is the question worth asking someone else in your organisation.

Who will be able to read it? Anyone with the content:read scope on the admin plane. Check that list before, not after.

If you turn it on, prefer a short ttl over the default. Thirty days of prompt text is a lot of prompt text, and the sweeper only removes what the lifetime says it may.

How capture relates to redaction and to the ledger's other content controls

Three different mechanisms touch content, and they are simple to confuse.

ledger.redact.prompt decides what the ledger row itself carries about the prompt. The shipped setting is hash.

dashboard.capture decides whether full bodies are written to the sidecar store. This page.

retention rotates and prunes the decision log itself, leaving the active file alone. The decision ledger on disk owns it.

They compose rather than override: capture off with redaction hashing is the default, and each can be changed without the other.

Turning it off again

Set both sides to false and restart. That stops new bodies being written; it does not delete the ones already there. To remove those, delete the store at capture.path, or set a short ttl and let the sweeper do it — the sweep is audited, which a manual delete is not.

Nothing about the ledger changes either way, because the bodies were not in it.

What can go wrong

Capture is on and records show no content. Most likely the traffic is streaming. Check a non-streaming request before suspecting the setting.

The panel says capturing and you did not turn it on. tokentriage demo enables both sides, because its traffic is synthetic and there is nothing to protect. A demo instance is not evidence about your own config.

The store grows faster than expected. It holds bodies, so it scales with prompt size rather than request count. Shorten the ttl.

Somebody can read prompts who should not. That is a role question rather than a capture question. content:read is privileged; audit who holds it.