# CLI reference

Every TokenTriage command group and its flags, enumerated from the binary's own command tree — proxy and serving, ledger analysis, routing policy, governance, and operations.

Source: https://tokentriage.com/docs/reference/cli

---
import Callout from '../../../components/Callout.astro';
import NextSteps from '../../../components/NextSteps.astro';

The binary ships 21 command groups (plus Cobra's `completion` and `help`). Every command and
flag below is taken from the command tree in `cmd/tokentriage`.

<Callout type="note">
The README's short command list is partial. This page is the complete, code-derived set.
Two things worth knowing: there is **no `--groq` run flag** (adding Groq is a config-only
change — see [Providers](/docs/build/providers)), and `keys mint` takes `--owner`, not
`--node`.
</Callout>

## Proxy & serving

| Command | Purpose & flags |
|---|---|
| `run` | Start the proxy. `-c, --config <F>`; `--anthropic` (ledger quickstart); `--simple <upstream/model>` and `--complex <upstream/model>` (a two-tier route quickstart); `--no-dashboard`. |
| `validate` | Strictly validate a config file and print a summary. `-c, --config <F>` (default `tokentriage.yaml`). Runs the real provider + policy compile, exactly as boot does. |
| `wrap` | Exec a command pointed at the proxy with a fresh per-session trace header. `--agent <NAME>` (sets `x-tt-agent`); `--base <URL>` (default `http://127.0.0.1:8787`); everything after `--` is exec'd. |
| `version` | Print build info and the embedded pricing snapshot. |

**Diagnostics repeat, and one fault can look like two.** Every command here except `doctor` prints
its fatal error twice, once by the command framework and once by the top-level handler; `doctor`
silences the framework's copy. `validate` also runs the structural matrix and then the registry
compile, so a warning both passes produce is printed once per pass. Count the distinct config paths
a message names, not the lines.

## Ledger analysis

| Command | Purpose & flags |
|---|---|
| `top` | Ledger rollups as terminal tables. `--log <F>` (default `~/.tokentriage/decisions.jsonl`); `--by model\|trace\|agent\|step-class\|tool` (default `model`); `--trace <ID>` (per-step cost tree); `--since <window>` (`24h`, `30d`, or an RFC3339 timestamp). |
| `report` | Shadow-savings report — would-route deltas by step-class and rule. `--log <F>`; `--since <window>`. |
| `eval` | Honesty metrics — does routing beat the zero-router baseline? `--log <F>`; `--since <window>`; `--quality oracle\|outcome\|shadow` (default `oracle`). |
| `calibrate` | Fit / promote a versioned calibration artifact. `-c, --config <F>`; `--log <F>`; `--dir <D>`; `--target-frontier-pct <0..1>`; `--quality-budget <0..1>` (mutually exclusive with the former); `--activate <id>`; `--rule <id>` (legacy) or `--param <name>` (v2); `--min-labeled <int>`. A fit writes a candidate only — nothing routes until `--activate`. |
| `prices` | Inspect the embedded pricing snapshot. `prices show <MODEL>`; `prices diff <newer.json>`. |
| `cache preview` | Read-only replay estimate of exact-tier repeat-rate + would-save. `--log <F>`; `--since <window>`. See [Response cache](#response-cache) below. |

## Quality calibration

The `quality` group is the lifecycle of the `qcal_` **quality-calibration** artifact — the
[calibration engine](/docs/calibration/overview) that turns an evaluator's raw score
into a probability whose error has been measured on a held-out slice. Persistent flags apply to
every subcommand: `-c, --config <F>`; `--log <F>` (default `~/.tokentriage/decisions.jsonl`);
`--dir <D>` (the store, default `<state_dir>/calibrations/quality`); `--state-dir <D>` (holds
`audit.jsonl`, the calibration trail).

| Command | Purpose & flags |
|---|---|
| `quality calibrate` | Fit every `(segment × evaluator)` unit from the joined decision log on a time-ordered **50/25/25** split, conformalize, measure error on the untouched holdout, and write an **inert candidate** artifact. Demo and synthetic/dataset rows are excluded and counted. `--confidence <0..1>` (nominal coverage; defaults to `evals.calibration.quality.confidence`); `--min-n <int>` (per-segment floor — **raise-only**; a flag that would lower the evidence floor is refused); `--since/--until <window>`; `--fingerprint <epoch>` (the policy epoch — a fit refuses without one, deriving it from `--config` when absent); `--now <RFC3339>` (pin the clock for a reproducible id). |
| `quality activate <artifact-id>` | Promote a candidate/shadow artifact to **active**, atomically retiring the incumbent. A gate-failed artifact is **refused with its reliability report attached**; a mixed artifact activates with its failing segments serving nothing. Audited — success, refusal, and failure each write one row. |
| `quality reliability [artifact-id]` | Print the stored B.5 reliability report verbatim (per-segment error, threshold and its two terms, holdout size, gate verdict). Default: the active artifact. A read — writes no audit row. |
| `quality coverage` | The B.9 evidence-coverage meter: `licensed rows / applicable rows` per metric and overall, with the biggest uncovered segments' need/have gaps. The ratio is `null`, never `0.0`, when nothing is applicable. `--metric <id>`; `--top <n>`; `--since/--until <window>`; `--json` (the exact bytes `GET /tt/api/v1/quality/coverage` serves). |

<Callout type="warning">
**`tokentriage quality calibrate` is not `tokentriage calibrate`.** They ride the same lifecycle
chassis but write different **kinds** of artifact. `quality calibrate` (this group) fits a
**probability** calibration — how good outputs are on a metric — into `<state_dir>/calibrations/quality`.
`calibrate` (under [Ledger analysis](#ledger-analysis)) fits a **routing threshold** — what share
of traffic goes to the frontier tier — into `~/.tokentriage/calibrations`. Always name which one.
See [The calibration lifecycle](/docs/calibration/lifecycle) for the shared state machine.
</Callout>

## Evals

The `evals` group drives the offline eval program: versioned datasets of **ledger references**
(never copies of rows) re-scored by named instruments, projected onto a cost/quality Pareto
frontier and a four-state verdict. Persistent flags: `-c, --config <F>`; `--log <F>`; `--dir <D>`
(the dataset store, default `<state_dir>/evals/datasets`); `--state-dir <D>`; `--content-dir <D>`
(the captured-body store; absent bodies are skipped-and-counted, never scored).

| Command | Purpose & flags |
|---|---|
| `evals datasets [dataset-id]` | List dataset artifacts, or show one with its fit-eligibility and item census (synthetic/imported rows are structurally fit-ineligible and it says so). `--json`; `--resolve` (re-resolve the snapshot against the live ledger and report members retention rotated out); `--items <n>` (how many `req_id`s to print). |
| `evals run <dataset-id>` | Execute a `judged_dataset_run` — evaluators re-score the **stored outputs** of the referenced rows — and print the run-history entry, refusals included. Every-item-uncaptured is a **counted refusal** naming `config.DashboardCapture.response`, never an empty success. `--arm <spec>` (repeatable); `--arm-config <arm>=<json>`; `--run-id`; `--experiment`; `--title`; `--segment`; `--now <RFC3339>`; `--out <F>` (write the whole run record); `--json`. |
| `evals compare <dataset-id>` | The same execution projected onto the comparison surface: arms table, cost/quality Pareto frontier, and the four-state verdict (`BEAT` / `NOT_BEAT` / `ON_FRONTIER` / `CANNOT_MEASURE`) with its reason. `--baseline <arm>`; `--candidate <arm>`; `--run <F>` (read a saved run instead of re-executing); `--json` (the exact bytes `GET /tt/api/v1/evals/experiments/{id}` serves); plus all `run` flags. |
| `evals validate` | Assemble every gold set the decision log supports, run the B.7 judge-validation protocol for each judge subject, and record every run in the `jval_` artifact store beside the promotion registry. A declined run persists exactly like a promotion and names the floor it missed with the need/have that missed it. `--json` (the same runs as the wire shapes `GET /tt/api/v1/evals/judges` serves); `--now <RFC3339>` (the instant to stamp the records, the artifact and the recalibration entries with; default now, UTC — a value at or before the newest stored artifact is refused, and an unparseable one exits 1). This is an **operator** run and does not consult `evals.validation.auto_validate.enabled`, which schedules the automatic loop; asking by hand is always allowed. |

**Arm syntax** (`--arm`): `name=<arm>,evaluator=<registered id>,metric=<registered id>[,tier=<tier>][,basis=measured|reported][,instrument=judged_dataset_run|replay]`.

The verb is `evals validate`, and there is no `evals judges`. Typing one prints the `evals` group
help and exits **0**, as does any other unrecognized `evals` subverb; only an unrecognized
*top-level* verb exits 1 with `unknown command`. [Validate a judge and promote
it](/docs/judges/validate-and-promote) walks a declined run and a promoted one side by side, and
[Evals & calibration configuration](/docs/reference/evals-config) documents the
`evals.validation` leaves this run reads.

<Callout type="note">
The CLI **refuses external evaluators by name** — a `judge`/`sidecar` needs the `external: allow`
air-gap gate and a key env-ref that this command has no config surface for. It also refuses the
`shadow_dual_run` instrument (the only one whose verdict can be a `BEAT`), because a live
counterfactual needs a running data plane and a spend authorization — drive it from
`POST /tt/api/v1/evals/experiments/{id}/run`. A `judged_dataset_run` re-scores outputs that
already happened, so its best reachable verdict is `ON_FRONTIER`. See
[Eval programs & verdicts](/docs/calibration/eval-programs).
</Callout>

## Routing policy

| Command | Purpose & flags |
|---|---|
| `policy migrate` | Print the desugared v2 `routing.policy` a legacy `routing.rules` config compiles to. `-c, --config <F>`. |
| `policy explain` | Render one logged decision's explanation. `--log <F>`; `--req-id <ID>`. |
| `policy preview` | Offline re-decide of the log under a draft policy (with mandatory approximation labels). `-c, --config <F>` (the draft); `--log <F>`. |

## Governance

| Command | Purpose & flags |
|---|---|
| `keys mint` | Mint a virtual key (secret shown once). `--owner <node>` (e.g. `org:acme/team:ml`); `--label`; `--models`, `--providers`, `--endpoints` (CSV scopes; empty = unrestricted); `--expiry <dur>`; `--allow-passthrough`. |
| `keys list` / `keys revoke <KID>` / `keys rotate <KID> --grace <dur>` | List (never the secret) · revoke (terminal) · rotate (mint a successor). |
| `govern preview` | Dry-run a proposed `governance.limits` set over the log. `-c, --config <F>`; `--log <F>`; `--json`; `--max-rows <int>`. |
| `govern reconcile` | Report spend drift vs a claimed rollup (never adopts). `-c`; `--log`; `--rollup <file.json>`; `--as-of <RFC3339>`; `--json`. |
| `user …` | Manage local dashboard users (argon2id). `user add <NAME> --role viewer\|operator\|admin`, `user list`, `user remove`, `user passwd`, `user set-role`. Persistent `--state-dir`. |
| `token …` | Manage dashboard bearer tokens (SHA-256, shown once). `token new --role … --label … --ttl <dur>`, `token list`, `token revoke <ID>`. Persistent `--state-dir`. |

<Callout type="warning">
`keys mint` and `token new` (and `keys/*`, `user/*`) persist state under `--state-dir`
(default `~/.tokentriage/dash`), which holds `governance.db` / `tokens.json` — the SHA-256
hashes, never the secrets. The plaintext is printed once and never again.
</Callout>

## Hosted runtime credentials

The `cloud` group addresses the hosted customer lane rather than a local proxy, so every command in
it takes `--base-url <origin>` (no default) and reads a management key from
`TOKENTRIAGE_MANAGEMENT_KEY` or `--management-key-stdin`. No flag in the group accepts a credential
value as an argument. `cloud credential` manages the runtime keys an application's workload sends;
[your first routed request](/docs/sdks/first-request) is what one is for.

| Command | Purpose & flags |
|---|---|
| `cloud credential create` | Mint a runtime credential for a service identity. The value is printed once, on stdout, and the server keeps a digest. `--service <svc_…>`; `--env <env_…>`; `--gateway <gw_…>` (repeat for each, at least one); `--label` (required); `--ttl-days <int>` (at most 365); `--team <team_…>` (repeat for each, at most 8; needs `members.manage`); `--wait` with `--wait-timeout <dur>` (default 30s, ceiling 5m). A repeated create mints a second credential; there is no `--idempotency-key`. |
| `cloud credential rotate` | Replace a credential's secret, keeping its service identity (`svc_…`). New value printed once. `--credential-id <kid_…>`; `--overlap-minutes <int>` (default 15, range 0–1440) **or** `--revoke-previous-now`, which contradict each other; `--team <team_…>` (repeat for each; replaces the credential's teams) **or** `--no-team` (clears them), which contradict each other; `--wait`. With neither team flag the replacement carries the predecessor's teams. The replacement keeps the credential's label and lifetime policy. |
| `cloud credential revoke` | Revoke a credential. `--credential-id <kid_…>`. New dispatch stops within 30 seconds; calls already in flight may finish and may cost money. A second revocation answers 409. |
| `cloud credential status` | Read one mint or rotation's readiness. `--operation <ID>` from the document `create` or `rotate` printed. `readiness: ready` is the only state in which the credential is known to work. |

A minted credential is not a working one: it works once the data-path grant has been acknowledged,
which is what `--wait` and `status` report. All four verbs need `keys.manage` in the organization,
and `cloud service-identity list|create|get` manages the identities credentials are issued to.

Team assignment (`--team`, `--no-team`) needs `members.manage` as well as `keys.manage`, and it must be
enabled on the deployment (`TTCP_FOUNDATION_KEY_TEAMS`); where it is not, those flags are refused.
A rotate with no team flag carries the existing teams and always works.

## Guardrails

| Command | Purpose & flags |
|---|---|
| `guardrails list` | Every registered detector with its coverage, **its gaps**, and the license of any data it embeds. `--config <F>` additionally prints each detector's calibration state. This is the source of truth for what the running binary can find. |
| `guardrails explain` | Resolve ONE configured binding against the **compiled** engine: the detectors it arms and their honest scope, the threshold and severity bands, the resolved ladder rung by rung, and whether this build can carry each verb out. `--binding <ID>` (required); `--config <F>` (required); `--content <TEXT>` to also show what it would do to that text. No traffic. |
| `guardrails test` | Dry-run the configured guardrails over content you supply. **Enforces nothing**: no byte changed, no ledger row written, no traffic refused; the report is stamped `mode: simulate` with a separate `enforced: false` bit. Name exactly one subject: `--content`, `--content-file`, `--req-id`, `--dataset`, or `--corpus`. `--config <F>` required. |

<Callout type="warning">
`guardrails test` runs **embedded detectors only** — judge- and sidecar-locality detectors are
skipped and listed under `SKIPPED`. A dry run that billed per corpus case is a surprise on an
invoice, and an operator endpoint that can be made to dial out is a weaker air-gap posture. Its
latency figures describe the selected operation on that host, not a production floor or load
prediction. Read `operation_kind`: literal diagnostics and structured policy dry runs are distinct. See
[Proving it before you arm it](/docs/guardrails/proving-it-before-you-arm-it).
</Callout>

## Operations

| Command | Purpose & flags |
|---|---|
| `demo` | Boot the dashboard against a synthetic, watermarked ledger with live mock traffic. `--open`; `--rows <int>` (default 40); `--live <int>` (default 6); `--rich`; `--heartbeat <dur>`; `--admin <host:port>`; `--data <host:port>`. Ephemeral loopback ports unless bound. |
| `doctor` | Read-only setup diagnostics (never boots, never writes, never prints a secret). `-c, --config <F>`; `--json`; `--probe`; `--probe-url <URL>`. |
| `index rebuild` | Rebuild the derived SQLite read-model from the log. `--index <path>`; `--log <F>`. |
| `fleet list` / `fleet decommission <name>` | Inspect / manage the durable replica roster. Persistent `--state-dir`, `-c, --config`. |

## Response cache

The `cache` command group holds exactly **one** CLI subcommand — the read-only
`cache preview` estimator (also listed under [Ledger analysis](#ledger-analysis) above).

| Command | Purpose & flags |
|---|---|
| `cache preview` | Labeled, replay-based estimate over the decision log: the EXACT-tier repeat-rate + would-save, plus the SEMANTIC-tier content-capture coverage (or an honest `CANNOT-ESTIMATE`). Read-only over historical rows. `--log <F>` (default `~/.tokentriage/decisions.jsonl`); `--since <window>` (`24h`, `30d`, `2w`, or an RFC3339 timestamp). |

<Callout type="note">
`cache preview` is the **secondary** instrument. Shadow mode (`cache.rules[].mode: shadow`)
is the primary measurement — real lookups on live traffic. Every figure `cache preview`
prints carries its approximation labels (`fixed_usage`, `recorded_runtime_state`,
`fingerprint_approximated`); the exact-tier repeat counts are an **upper bound** because the
ledger stores a messages-only prompt hash, not the full fingerprint.
</Callout>

### Cache stats, evidence, and purge (admin API — not CLI)

There is **no `cache stats`, `cache evidence`, or `cache purge` subcommand**. Those
surfaces live on the running proxy's dashboard admin API, because they read live cache
state (or physically evict entries) — something a standalone CLI process cannot do. They are
served under the `/tt/api/v1` prefix and require an authenticated dashboard token or session.

| Method & path | Role floor · scope | Purpose |
|---|---|---|
| `GET /tt/api/v1/cache/stats` | viewer · `cache:read` | Outcome counts (`hit`, `would_hit`, `miss`, `bypass`, `error`), per-kind hit counts, and the **hit-only** licensed savings beside the separately-labeled `shadow_savings` counterfactual (the two are never merged). |
| `GET /tt/api/v1/cache/evidence` | viewer · `cache:read` | The verdict surface a licensed savings figure's `verdict_ref` points at: per-kind licensing posture (`licensed` \| `cannot_measure`), the reason for any refusal, and the stored-entry age distribution. |
| `GET /tt/api/v1/cache/preview` | viewer · `cache:read` | The HTTP form of the `cache preview` estimate — the same labeled, secondary pre-enablement figure over historical rows. Accepts no `by`/`value`; it is **not** a dry-run of purge. |
| `POST /tt/api/v1/cache/purge` | operator · `cache:write` | Invalidate cached entries along a closed dimension. Body `{ "by": "entry"\|"scope"\|"rule", "value": "<EntryID\|ScopeKey\|rule id>" }`; returns `{ by, value, deleted }` — the count of entries **physically** removed. Destructive, has no dry-run, and writes exactly one `cache.purge` audit row on every branch (accepted, rejected, or failed). |

<Callout type="warning">
`POST /tt/api/v1/cache/purge` is destructive and cannot be rehearsed — there is no operation
that reports which entries a given `by`/`value` would remove. The audit row makes a purge
**attributable** after the fact (who, which dimension, the value, and the resulting count),
but not reconstructable: for the bulk `scope` / `rule` dimensions it does not record *which*
individual entries died. When the cache is disabled or the purge seam is unwired, all four
endpoints answer `404 feature_absent` naming the config that enables them — never a fabricated
success.
</Callout>

The full config schema those commands validate is the
[Configuration reference](/docs/reference/configuration).

<NextSteps items={[
  { title: "Calibration & Evals", href: "/docs/calibration/overview", description: "What sits behind the quality and evals command groups — estimates, calibration, and honest verdicts." },
  { title: "Evals & calibration configuration", href: "/docs/reference/evals-config", description: "The full evals: config block the quality and evals commands read — evaluators, the air-gap gate, and the calibration knobs." },
  { title: "Configuration reference", href: "/docs/reference/configuration", description: "The full config schema — including the cache: block these commands validate against." },
  { title: "Observability", href: "/docs/build/observability", description: "The dashboard and admin API that serve cache stats, evidence, and purge." }
]} />