Redact PII before it reaches the model

Goal: replace detected structured PII in supported prompt fields before provider dispatch, then measure false positives before enforcement. Successful inspection and transformation are required; this example inherits on_error: open, which may forward original bytes when protection fails. Use acknowledged closed enforcement when that continuation is unacceptable.

1. Pick the detectors deliberately

The nine embedded pii.* detectors are individually bindable, and arming all nine is not the safe default it looks like.

guardrails:
  enabled: true
  default_profile: standard
  local_budget_ms: 50
  guards:
    - id: pii_in
      category: pii
      phases: [input]
      detectors: [pii.card, pii.email, pii.iban, pii.ssn, pii.phone]
      mode: monitor
      action: redact
      rule_ref: POL-PII-1
  profiles:
    standard: [pii_in]

Read what each one misses before arming it. Three that catch people out: pii.phone needs a leading + and misses every national format; pii.ssn is US-only and misses a bare nine-digit run in prose; pii.email deliberately does not refuse example.com.

2. Dry-run it before it sees traffic

$ tokentriage guardrails test --config tokentriage.yaml \
    --content 'Please charge card 4111 1111 1111 1111 for [email protected]'

Read operation_kind, case inspection state and execution records. Literal text defaults to detector_diagnostics; it is not a serving-path proof.

Point it at authorized stored traffic with --req-id or --dataset to run over stored request bodies. Use --corpus if you have a labeled set — that is the only subject that carries ground truth.

3. Run it in `monitor` and read the rows

monitor runs the detectors on real traffic and changes no byte. Every fire lands on the row with effect: annotated.

import json, time, urllib.request

# The guard runs on every request; you read what it did from the decision row.
req = urllib.request.Request(
    "http://127.0.0.1:8787/v1/messages",
    data=json.dumps({
        "model": "claude-haiku-4-5",
        "max_tokens": 64,
        "messages": [{"role": "user",
                      "content": "Charge card 4111 1111 1111 1111 for [email protected]"}],
    }).encode(),
    headers={"content-type": "application/json"},
)
with urllib.request.urlopen(req) as response:
    response.read()  # Complete the response before looking for its row.
    req_id = response.headers["x-tt-request-id"]
if not req_id:
    raise RuntimeError("gateway request ID missing")

# Primary rows omit kind; companions may be appended afterward. Match this request.
deadline = time.monotonic() + 5
row = None
while time.monotonic() < deadline:
    try:
        with open("decisions.jsonl", encoding="utf-8") as fh:
            for line in fh:
                if not line.endswith("\n"):
                    continue  # An append may still be in progress.
                candidate = json.loads(line)
                if candidate.get("kind") is None and candidate.get("req_id") == req_id:
                    row = candidate
    except FileNotFoundError:
        pass
    if row is not None:
        break
    time.sleep(0.05)
if row is None:
    raise RuntimeError("primary decision row not available within five seconds")

print(json.dumps(row["disclosures"]["guardrails"]["fired"], indent=2))

import { readFileSync } from "node:fs";

const response = await fetch("http://127.0.0.1:8787/v1/messages", {
  method: "POST",
  headers: { "content-type": "application/json" },
  body: JSON.stringify({
    model: "claude-haiku-4-5",
    max_tokens: 64,
    messages: [{ role: "user",
                 content: "Charge card 4111 1111 1111 1111 for [email protected]" }],
  }),
});

await response.arrayBuffer(); // Complete delivery before looking for its row.
if (!response.ok) throw new Error(`gateway returned HTTP ${response.status}`);
const reqId = response.headers.get("x-tt-request-id");
if (!reqId) throw new Error("gateway request ID missing");

// Primary rows omit kind; match this request and leave all companion rows intact.
const deadline = Date.now() + 5000;
let row: any;
while (Date.now() < deadline) {
  let ledger = "";
  try { ledger = readFileSync("decisions.jsonl", "utf8"); }
  catch (error) { if ((error as NodeJS.ErrnoException).code !== "ENOENT") throw error; }
  const completeLines = ledger.split("\n").slice(0, -1);
  row = completeLines.filter(Boolean).map((line) => JSON.parse(line))
    .find((candidate) => candidate.kind == null && candidate.req_id === reqId);
  if (row) break;
  await new Promise((resolve) => setTimeout(resolve, 50));
}
if (!row) throw new Error("primary decision row not available within five seconds");

console.log(row.disclosures.guardrails.fired);

{
  "binding": "pii_in",
  "detector": "pii.card",
  "detector_version": "pii.card-v1",
  "action": "redact",
  "effect": "annotated",
  "entities": [{ "type": "CREDIT_CARD", "count": 1 }],
  "reversible": false,
  "confidence": { "basis": "exact", "point": 1 },
  "rule_ref": "POL-PII-1"
}

4. The false-positive check — do not skip this

$ curl -s http://127.0.0.1:9090/tt/api/v1/guardrails/evidence |
    jq '{adjudicated: .quality_sources, silent: [.configured[] | select(.fired==false) | .id]}'

Three things to satisfy yourself about before climbing:

Adjudicated outcomes above zero, with a denominator you would defend.
suspended[] is empty on this binding. A guard that was silently not running is not a guard you have evidence about.
The binding is not in the “fired on nothing” roster. If it is, you have proved nothing yet.

5. Enforce

      mode: enforce

When supported inspection and transformation complete, redact splices the detected span and the model sees [REDACTED:CREDIT_CARD] instead of the PAN. Verify the execution, transformation and delivery receipts as well as effect: mutated. The default open failure posture can still forward original content when protection cannot complete.

Rollback is one line back to monitor. Nothing else changes: the same detectors keep running and the same rows keep being written.