# Reversible de-identification The tokenize and re-identify round trip — what the vault is, which verbs are reversible, and the failure modes that matter. Source: https://tokentriage.com/docs/guardrails/reversible-deidentification --- import Callout from '../../../components/Callout.astro'; import NextSteps from '../../../components/NextSteps.astro'; import GuardrailVaultRoundTripDiagram from '../../../components/GuardrailVaultRoundTripDiagram.astro'; `redact` and `mask` throw the value away: the model answers about `[REDACTED:EMAIL]` and so does your client. **Reversible de-identification keeps the answer usable.** The input gate substitutes a placeholder, the model reasons about that, and the output gate puts the real value back before your client sees the response. ## The round trip Configure the **forward** verb only. Measured on a live proxy with `action: tokenize` bound to `pii.email` on the input phase: ```console # what the client sent "my email is ada@example.com" # what the upstream actually received "my email is [TTsbjk:EMAIL_1]" # what the client got back "Sure — I have noted my email is ada@example.com and will follow up there." ``` Both legs are on the decision row — the forward verb with its detector and finding, the reverse leg without one, because no detector runs on it: ```json {"binding":"pii_tokenize_in","phase":"input","detector":"pii.email","action":"tokenize", "findings":[{"entity":"EMAIL","spans":[{"field":"messages[0].content","off":12,"len":15}], "raw_score":0.85,"confidence":{"basis":"reported","point":0.85}}],"mode":"enforce"} {"binding":"pii_tokenize_in","phase":"output","detector":"","action":"reidentify","mode":"enforce"} ``` **`reidentify` is not a verb you configure.** It is minted by the engine as the reverse leg of a `tokenize` or `pseudonymize` round trip, and a config naming it is refused at boot: ```console Error: binding "reid_out": action "reidentify" is the engine-internal reverse leg of a tokenize round-trip, never a configured action — it is MINTED BY THE ENGINE […] this build implements no span applier for it, so a ladder naming it would splice NOTHING and disclose `annotated` over content it was configured to change. Configure the FORWARD verb (`tokenize` or `pseudonymize`) and the reverse leg runs on the output phase by itself ``` ## `tokenize` or `pseudonymize` Both are `Mutates: true, Reversible: true` and both round-trip identically. They differ only in what the model sees, and the same request under each verb shows why you would pick one: | Verb | What the upstream receives | Pick it when | |---|---|---| | `tokenize` | `[TTsbjk:EMAIL_1]` | the model must not mistake the surrogate for real data | | `pseudonymize` | `grace.jones.rlyn7gq0@example.net` | a realistic-looking value reads better to the model | Within one request the same original always maps to the **same** placeholder — that is entity consistency, and it matters: a prompt mentioning one person twice must not read as two people. Measured: ```console "ping grace@example.com and grace@example.com again" ⇒ "ping [TTj5xz:EMAIL_1] and [TTj5xz:EMAIL_1] again" ``` A `pseudonymize` surrogate can fall back to the bracketed form, and the fire still discloses `pseudonymize` — the verb the policy chose. What degraded is the **realism** of the surrogate, not the action taken; the reversible and mutating bits are identical for the two verbs, so nothing an auditor reads changes meaning. ## Which verbs are reversible, and why that bit is trustworthy `Reversible` is not authored per fire. It is **derived** from one canonical action table, which is also what validates a verb in the first place — so the table cannot drift from the disclosure: | Verb | Reversible | |---|---| | `tokenize` · `pseudonymize` · `reidentify` | **yes** | | `redact` · `mask` · `rewrite` · `block` · `warn` · `observe` · `reask` | no | This table describes the canonical verb taxonomy, not gateway availability: stock activation rejects `rewrite` and `reask`. The lower-level Go re-ask callback exists for custom embeddings. **`redact` and `mask` are irreversible and no configuration changes that.** If you need the answer to come back intact, the choice is made at the forward verb. `guardrails explain` prints the resolved ladder with both bits, so you can check before arming: ```console LADDER — the verb each severity resolves to (fall-down included) SEVERITY ACTION REVERSIBLE THIS BUILD CAN DO IT low tokenize yes yes ``` ## The vault is per-request The mapping lives on **one request's state and dies with it.** It is minted on first reversible need, never before, and there is no package-level vault, no registry of vaults, and no constructor that takes one from anywhere but a request. The anti-pattern this avoids by construction is a process-wide vault shared across clients — a cross-tenant re-identification pool. Each phase mints its own. The output phase used to be seeded with the input phase's vault, which made the reverse leg undo the output binding's own work on the same request. **`vault.key_env` does not make anything durable in this build.** It names an environment variable holding a base64 AES-256 key, and setting it makes the record codec *usable* — that is the complete list for the vault codec. **The vault mappings are not persisted, and no production code path calls their sealing routines.** This does not describe the ordinary decision ledger or other content sinks, which retain their own storage and access policies. It also does **not** make `pseudonymize` surrogates stable across requests: a surrogate is derived from a per-request nonce whether or not a key is set. The key changes the derivation's secret, never its per-request-ness. Like every other credential in the product, the config **names** the variable and never carries the value — the same discipline described in [Security & secrets](/docs/operate/security). ```yaml guardrails: vault: key_env: TT_VAULT_KEY # the NAME of the variable, never the key ``` ## Two acknowledgement keys, and what they actually do - **`raw_egress_ack`** is retained compatibility syntax. It does not authorize supported raw egress in this build: stock activation rejects every external `NeedsRaw` binding even with the ack. Destination-specific external consent remains a separate requirement for supported external calls. - **`fuzzy_reid`** is accepted, compiled onto the binding, and **read by nothing.** **`fuzzy_reid` changes no behavior in this build, and that is a decision rather than a deferral.** The reverse leg is exact-match only. A fuzzy match can re-identify the *wrong* span, and fuzzy re-identification is a named anti-pattern in this layer's own source review. Setting the key gives you nothing; leaving it unset costs you nothing. ## The failure modes The reverse leg mutates, so it discloses. What matters is what it does when it *cannot* complete — and in every case the answer is a safe failure rather than a fabricated clean round trip. **A placeholder the model mangled is not restored.** If the model drops a bracket or splits the placeholder across a word boundary, the exact match fails and the mangled marker **survives to the client verbatim.** That is the safe failure: an unreadable marker, never someone else's data. The round-trip corpus measures how often it happens rather than papering over it. **A cached response is never re-identified.** A cache hit's bytes are some earlier request's answer; nothing in them was tokenized by *this* request, so there is nothing of ours to restore — and restoring anyway is not a no-op, it is a cross-request leak. This was measured, not theorized: under an early process-wide nonce, a stored body reading `the answer about [TT0000:DRILL_TOKEN_1] is 42` was served to a later client as *the answer about that client's own secret* **is 42, with the row disclosing a clean round trip** — fabricated provenance on top of another request's cached answer. The per-request nonce that landed later **did not remove the need for that gate**, and it is worth being precise about why. The nonce is 4 base-36 characters drawn per request; its stated purpose is to keep a placeholder from colliding with genuine content. It is **not** a uniqueness guarantee — 36⁴ is 1,679,616 values, so two requests share a nonce after roughly 1,600 of them. What isolates requests is not probabilistic: the reverse leg searches **only** for strings this vault minted, and resolves each through this vault. That is a search-set property. A stronger nonce would not license removing the rule, and a weaker one is not what broke it. **On the streaming lane the reverse leg is not implemented.** The action table licenses `reidentify` at the stream phase, but the restoring code is the buffered output gate's. A cross-frame placeholder needs the hold-back window, and that is a separate seam — see [Phases & streaming](/docs/guardrails/phases-and-streaming).