Roll out a guard without breaking traffic

A runbook for taking one binding from “written down” to “changing production bytes” without a surprise. Each rung has an exit criterion, a watch-list, and a rollback that is the rung below it.

Rung 0 — before anything, decide the failure posture

Two settings decide what happens when the guard cannot run, and both belong to this decision rather than to a later one.

      on_error: open          # open (default) | closed

In enforcing mode, open discloses protection failures and may forward original bytes. closed requires a fail_closed_ack and withholds affected unreleased bytes when protection cannot complete. Monitor mode preserves traffic under either posture.

Rung 1 — guardrails explain

$ tokentriage guardrails explain --binding pii_in --config tokentriage.yaml

Exit criterion: the LADDER table shows THIS BUILD CAN DO IT: yes on the rung(s) your traffic will actually reach, and the DETECTORS section lists the detectors you intended on the phases you intended.

What to watch for:

• Unsupported stock actions are rejected during activation, including rewrite and reask. Inspect the capability errors before treating a ladder as executable.

• An empty detector list is rejected. format has no detector and custom has only the drill.echo test stub; vocabulary membership does not provide production coverage.

• A detector whose threshold and severity_bands cannot discriminate. Where every finding is exact at 1.0 (media.inventory), or the verdict is boolean (injection.judge), the threshold is an on/off switch and the ladder collapses to one rung.

Rollback: edit the YAML. Nothing has run.

Rung 2 — guardrails test

$ tokentriage guardrails test --config tokentriage.yaml --req-id <a stored request>

Exit criterion: the fires you expected on the content you expected, with no NO LADDER RUNG column entries, and no binding you meant to exercise sitting in SKIPPED.

What to watch for:

• SKIPPED lists bindings not bound to the phase you ran. That is correct behavior, not a problem — but check the list, because a binding you thought you were testing may be in it.
• A dry run runs embedded detectors only. Judge and sidecar detectors are skipped deliberately, so if your policy leans on an external tier, this rung says nothing about it.
• Read operation_kind. Inline detector_diagnostics is a literal detector probe; structured stored-subject policy_dry_run exercises the selected profile’s policy planning. Neither proves production delivery or predicts latency under concurrent load. Per-key overrides are refused.

Rollback: edit the YAML. Still nothing has run against traffic.

Rung 3 — mode: monitor

      mode: monitor

This is the rung that matters, and the one people leave too early. The guard runs on real traffic and changes no byte.

Exit criterion — all four:

1. Adjudicated outcomes above zero on this binding, with a denominator you would defend to somebody who disagreed with you.
2. suspended[] clean for this binding across the window.
3. The binding is not in the “fired on nothing” roster.
4. input_ms / output_ms on real rows are within your latency budget — not an inference from the dry run.

What to watch, on GET /tt/api/v1/guardrails/evidence or the dashboard’s Honesty center:

# 1. are there any adjudicated outcomes at all?
$ curl -s $ADMIN/tt/api/v1/guardrails/evidence | jq '.quality_sources'

# 2. did anything suspend?
$ curl -s $ADMIN/tt/api/v1/guardrails/evidence | jq '{coverage, suspension_rows, uninspected_rows, suspensions}'

# 3. is anything configured and silent?
$ curl -s $ADMIN/tt/api/v1/guardrails/evidence |
    jq '[.configured[] | select(.fired == false) | .id]'

Put query 3 on a schedule. A binding that stops firing is indistinguishable from a clean cohort until you look.

Rollback: remove the binding from the profile, or set the profile to "". Nothing was changed, so there is nothing to undo — only the selected guardrail execution to disable; ordinary decision logging continues.

Rung 4 — mode: enforce

      mode: enforce

Exit criterion: none. This is the top.

What changes on the row: a successful action can record mutated or blocked. The action label is not proof of transformation or delivery. Read the execution stage, transformation state and delivery result together.

What to watch in the first hour:

• Expected mutations without committed transformations. Check the failed stage, reason and effective mode; do not infer success from a detector hit. Stock spanless mutation bindings are rejected during activation.
• invalid_span, scan_exhausted and stream-window failures. Invalid or incomplete mutation plans are not committed as successful whole transformations. Closed enforcement withholds unreleased output; bytes already released cannot be recalled.
• Your refusal/error rate, and whether actual wire bytes match the selected failure posture.

Rollback: one line, back to monitor. The same detectors keep running and the same rows keep being written; only the verbs stop being carried out.

What needs separate evidence

• Semantic media inspection: no embedded OCR, audio or video analysis is provided. A supported media-presence block can refuse a request; missing media seams follow mode and failure posture.
• Complete stream delivery: bounded retention, malformed framing and cancellation need actual client-byte and delivery-receipt checks. Closed mode can suppress unreleased bytes, but cannot recall earlier output. A larger local budget does not create an unbounded guarantee.
• Population detection quality: authored fixtures and local transport tests cannot replace independent labeled deployment evidence.

See Air-gap & known limits.