GitHubBook a demoStart routing
Browse docs

CLI reference

Every TokenTriage command group and its flags, enumerated from the binary's own command tree — proxy and serving, ledger analysis, routing policy, governance, and operations.

Last updated

On this page

The binary ships 21 command groups (plus Cobra’s completion and help). Every command and flag below is taken from the command tree in cmd/tokentriage.

Proxy & serving

Command Purpose & flags
run Start the proxy. -c, --config <F>; --anthropic (ledger quickstart); --simple <upstream/model> and --complex <upstream/model> (a two-tier route quickstart); --no-dashboard.
validate Strictly validate a config file and print a summary. -c, --config <F> (default tokentriage.yaml). Runs the real provider + policy compile, exactly as boot does.
wrap Exec a command pointed at the proxy with a fresh per-session trace header. --agent <NAME> (sets x-tt-agent); --base <URL> (default http://127.0.0.1:8787); everything after -- is exec’d.
version Print build info and the embedded pricing snapshot.

Ledger analysis

Command Purpose & flags
top Ledger rollups as terminal tables. --log <F> (default ~/.tokentriage/decisions.jsonl); --by model|trace|agent|step-class|tool (default model); --trace <ID> (per-step cost tree); --since <window> (24h, 30d, or an RFC3339 timestamp).
report Shadow-savings report — would-route deltas by step-class and rule. --log <F>; --since <window>.
eval Honesty metrics — does routing beat the zero-router baseline? --log <F>; --since <window>; --quality oracle|outcome|shadow (default oracle).
calibrate Fit / promote a versioned calibration artifact. -c, --config <F>; --log <F>; --dir <D>; --target-frontier-pct <0..1>; --quality-budget <0..1> (mutually exclusive with the former); --activate <id>; --rule <id> (legacy) or --param <name> (v2); --min-labeled <int>. A fit writes a candidate only — nothing routes until --activate.
prices Inspect the embedded pricing snapshot. prices show <MODEL>; prices diff <newer.json>.
cache preview Read-only replay estimate of exact-tier repeat-rate + would-save. --log <F>; --since <window>. See Response cache below.

Quality calibration

The quality group is the lifecycle of the qcal_ quality-calibration artifact — the measured-quality substrate that turns an evaluator’s raw score into a probability whose error has been measured on a held-out slice. Persistent flags apply to every subcommand: -c, --config <F>; --log <F> (default ~/.tokentriage/decisions.jsonl); --dir <D> (the store, default <state_dir>/calibrations/quality); --state-dir <D> (holds audit.jsonl, the calibration trail).

Command Purpose & flags
quality calibrate Fit every (segment × evaluator) unit from the joined decision log on a time-ordered 50/25/25 split, conformalize, measure error on the untouched holdout, and write an inert candidate artifact. Demo and synthetic/dataset rows are excluded and counted. --confidence <0..1> (nominal coverage; defaults to evals.calibration.quality.confidence); --min-n <int> (per-segment floor — raise-only; a flag that would lower the evidence floor is refused); --since/--until <window>; --fingerprint <epoch> (the policy epoch — a fit refuses without one, deriving it from --config when absent); --now <RFC3339> (pin the clock for a reproducible id).
quality activate <artifact-id> Promote a candidate/shadow artifact to active, atomically retiring the incumbent. A gate-failed artifact is refused with its reliability report attached; a mixed artifact activates with its failing segments serving nothing. Audited — success, refusal, and failure each write one row.
quality reliability [artifact-id] Print the stored B.5 reliability report verbatim (per-segment error, threshold and its two terms, holdout size, gate verdict). Default: the active artifact. A read — writes no audit row.
quality coverage The B.9 evidence-coverage meter: licensed rows / applicable rows per metric and overall, with the biggest uncovered segments’ need/have gaps. The ratio is null, never 0.0, when nothing is applicable. --metric <id>; --top <n>; --since/--until <window>; --json (the exact bytes GET /tt/api/v1/quality/coverage serves).

Evals

The evals group drives the offline eval program: versioned datasets of ledger references (never copies of rows) re-scored by named instruments, projected onto a cost/quality Pareto frontier and a four-state verdict. Persistent flags: -c, --config <F>; --log <F>; --dir <D> (the dataset store, default <state_dir>/evals/datasets); --state-dir <D>; --content-dir <D> (the captured-body store; absent bodies are skipped-and-counted, never scored).

Command Purpose & flags
evals datasets [dataset-id] List dataset artifacts, or show one with its fit-eligibility and item census (synthetic/imported rows are structurally fit-ineligible and it says so). --json; --resolve (re-resolve the snapshot against the live ledger and report members retention rotated out); --items <n> (how many req_ids to print).
evals run <dataset-id> Execute a judged_dataset_run — evaluators re-score the stored outputs of the referenced rows — and print the run-history entry, refusals included. Every-item-uncaptured is a counted refusal naming config.DashboardCapture.response, never an empty success. --arm <spec> (repeatable); --arm-config <arm>=<json>; --run-id; --experiment; --title; --segment; --now <RFC3339>; --out <F> (write the whole run record); --json.
evals compare <dataset-id> The same execution projected onto the comparison surface: arms table, cost/quality Pareto frontier, and the four-state verdict (BEAT / NOT_BEAT / ON_FRONTIER / CANNOT_MEASURE) with its reason. --baseline <arm>; --candidate <arm>; --run <F> (read a saved run instead of re-executing); --json (the exact bytes GET /tt/api/v1/evals/experiments/{id} serves); plus all run flags.

Arm syntax (--arm): name=<arm>,evaluator=<registered id>,metric=<registered id>[,tier=<tier>][,basis=measured|reported][,instrument=judged_dataset_run|replay].

Routing policy

Command Purpose & flags
policy migrate Print the desugared v2 routing.policy a legacy routing.rules config compiles to. -c, --config <F>.
policy explain Render one logged decision’s explanation. --log <F>; --req-id <ID>.
policy preview Offline re-decide of the log under a draft policy (with mandatory approximation labels). -c, --config <F> (the draft); --log <F>.

Governance

Command Purpose & flags
keys mint Mint a virtual key (secret shown once). --owner <node> (e.g. org:acme/team:ml); --label; --models, --providers, --endpoints (CSV scopes; empty = unrestricted); --expiry <dur>; --allow-passthrough.
keys list / keys revoke <KID> / keys rotate <KID> --grace <dur> List (never the secret) · revoke (terminal) · rotate (mint a successor).
govern preview Dry-run a proposed governance.limits set over the log. -c, --config <F>; --log <F>; --json; --max-rows <int>.
govern reconcile Report spend drift vs a claimed rollup (never adopts). -c; --log; --rollup <file.json>; --as-of <RFC3339>; --json.
user … Manage local dashboard users (argon2id). user add <NAME> --role viewer|operator|admin, user list, user remove, user passwd, user set-role. Persistent --state-dir.
token … Manage dashboard bearer tokens (SHA-256, shown once). token new --role … --label … --ttl <dur>, token list, token revoke <ID>. Persistent --state-dir.

Operations

Command Purpose & flags
demo Boot the dashboard against a synthetic, watermarked ledger with live mock traffic. --open; --rows <int> (default 40); --live <int> (default 6); --rich; --heartbeat <dur>; --admin <host:port>; --data <host:port>. Ephemeral loopback ports unless bound.
doctor Read-only setup diagnostics (never boots, never writes, never prints a secret). -c, --config <F>; --json; --probe; --probe-url <URL>.
index rebuild Rebuild the derived SQLite read-model from the log. --index <path>; --log <F>.
fleet list / fleet decommission <name> Inspect / manage the durable replica roster. Persistent --state-dir, -c, --config.

Response cache

The cache command group holds exactly one CLI subcommand — the read-only cache preview estimator (also listed under Ledger analysis above).

Command Purpose & flags
cache preview Labeled, replay-based estimate over the decision log: the EXACT-tier repeat-rate + would-save, plus the SEMANTIC-tier content-capture coverage (or an honest CANNOT-ESTIMATE). Read-only over historical rows. --log <F> (default ~/.tokentriage/decisions.jsonl); --since <window> (24h, 30d, 2w, or an RFC3339 timestamp).

Cache stats, evidence, and purge (admin API — not CLI)

There is no cache stats, cache evidence, or cache purge subcommand. Those surfaces live on the running proxy’s dashboard admin API, because they read live cache state (or physically evict entries) — something a standalone CLI process cannot do. They are served under the /tt/api/v1 prefix and require an authenticated dashboard token or session.

Method & path Role floor · scope Purpose
GET /tt/api/v1/cache/stats viewer · cache:read Outcome counts (hit, would_hit, miss, bypass, error), per-kind hit counts, and the hit-only licensed savings beside the separately-labeled shadow_savings counterfactual (the two are never merged).
GET /tt/api/v1/cache/evidence viewer · cache:read The verdict surface a licensed savings figure’s verdict_ref points at: per-kind licensing posture (licensed | cannot_measure), the reason for any refusal, and the stored-entry age distribution.
GET /tt/api/v1/cache/preview viewer · cache:read The HTTP form of the cache preview estimate — the same labeled, secondary pre-enablement figure over historical rows. Accepts no by/value; it is not a dry-run of purge.
POST /tt/api/v1/cache/purge operator · cache:write Invalidate cached entries along a closed dimension. Body { "by": "entry"|"scope"|"rule", "value": "<EntryID|ScopeKey|rule id>" }; returns { by, value, deleted } — the count of entries physically removed. Destructive, has no dry-run, and writes exactly one cache.purge audit row on every branch (accepted, rejected, or failed).

The full config schema those commands validate is the Configuration reference.