# Glossary Every term these docs use, in plain words. Start with the ten a newcomer needs, then look the rest up by area. Source: https://tokentriage.com/docs/reference/glossary --- This page defines every word these docs use that isn't ordinary English. Read the first section straight through, then look terms up as you hit them. ## Start with these ten Ten terms carry the rest of the documentation. Read them before anything past [What is TokenTriage?](/docs/introduction/what-is-tokentriage). - **Token** — the unit a model reads and charges in. One token is a few characters, so a short paragraph is a few hundred of them. [Where the money goes](/docs/introduction/the-cost-problem) - **Upstream** — the account and endpoint a request is actually sent to, such as your OpenAI account or your AWS Bedrock region. [Providers](/docs/build/providers) - **Tier** — a nickname you give one model at one provider, so rules can say `cheap` and `strong` instead of model strings. [Routing](/docs/build/routing) - **Dialect** — the shape of request and response body a provider expects. Every shipped dialect is recognised on its own paths; two of them, OpenAI's and Anthropic's, are also what cross-dialect translation can read from your code. [Core concepts](/docs/concepts/core-concepts) - **Data plane and admin plane** — two separate ports. `127.0.0.1:8787` takes your traffic; `127.0.0.1:9090` serves health, metrics and the dashboard. [Architecture](/docs/concepts/architecture) - **The decision ledger** — a text file with one line per request that went through, holding what was asked, what it cost and what was decided. Default path `~/.tokentriage/decisions.jsonl`. [Core concepts](/docs/concepts/core-concepts) - **Trust ladder** — the three settings TokenTriage runs in, ordered by how much it may change: `ledger`, then `shadow`, then `route`. [Core concepts](/docs/concepts/core-concepts) - **Shadow mode** — the middle setting. TokenTriage works out which model it would have picked and what that would have cost, writes both down, and sends your request untouched. [First routing rollout](/docs/get-started/first-routing-rollout) - **Provenance** — the fields stamped beside every dollar figure saying where it came from: which price snapshot, whether it was calculated or reported, and by whom. [Cost provenance](/docs/concepts/honesty-model) - **UNPRICED** — the mark on a request whose dollar figure is not known, either because the model has no price on file or because an async provider has not reported the money yet. The cost is left blank and the request counts as excluded, unlike a model priced at zero because it's free. [Where the money goes](/docs/introduction/the-cost-problem) The rest groups every other term by where you'll meet it, without repeating those ten. ## Running the proxy - **Proxy** — a server your application sends requests to instead of the provider. It passes them on and returns the reply. - **Gateway** — a proxy that also does something with the traffic on the way through: costing, logging, routing, capping. [LLM gateways explained](/docs/introduction/llm-gateways-explained) - **Base URL** — the address your SDK sends requests to. Point it at TokenTriage and the rest of your code is unchanged. - **Ledger mode** — the default. Every request is watched and costed; nothing about it changes. - **Route mode** — the top rung. Decisions take effect: the model name can be rewritten and a cheaper upstream chosen. - **Streaming** — the provider sends its answer in pieces as it's written, instead of one complete response. - **Context window** — the most tokens a model accepts in one request, counting your prompt and its reply together. - **Wire dialect** — a provider protocol the binary speaks. 11 wire-dialect drivers are compiled in: `openai`, `anthropic`, `bedrock`, `gemini`, `anthropic_vertex`, `elevenlabs`, `deepgram`, `assemblyai`, `replicate`, `soniox` and `stability`. The generator counts a driver where an `internal/dialect` package name equals the dialect id a descriptor names. [Wire dialects](/docs/build/providers-dialects) - **Provider descriptor** — the data defining a provider: authentication, price prefix, endpoints. TokenTriage ships 95 built-in provider descriptors, and adding an OpenAI-compatible one is a config change with no new Go code. Where a test drives a request through one, the generated matrix names that test; where none does, it prints the cell as ungated rather than hiding it, and ten descriptors carry no gated cell at all. The verification is against mocks, not against the providers' live endpoints. - **Cross-dialect translation** — rendering a request written in one client format into the shape a different upstream dialect expects. Two client formats, OpenAI's and Anthropic's, reach all four translation upstreams (`openai`, `anthropic`, `gemini`, `bedrock`), streaming and not, and each of the 12 translated cells is covered by an end-to-end test you can run against a mock shaped like that upstream. It's off until you turn it on, and any parameter that couldn't cross is listed on the row. [The dialect matrix](/docs/build/providers-dialect-matrix) - **`x-tt-*` headers** — the headers you set to say which trace, step, agent and tier a request belongs to. Identity is read from these rather than guessed. [Headers](/docs/reference/headers) - **Trace and step** — a trace is one unit of work, such as an agent run; a step is one model call inside it. Headers name both. - **Step-class** — what kind of step a call is: `user_turn`, `tool_loop`, or `provider_state`. Routing rules can match on it. - **Fail open** — when something inside TokenTriage goes wrong, your request is forwarded to the provider unchanged rather than failed. ## Cost and pricing - **Input tokens and output tokens** — what you sent and what came back. They bill at different rates, output usually dearer, so both are counted separately on every row. - **Prompt cache** — the provider's own reuse of a prompt prefix it has already processed. Writing costs a premium; later reads are discounted. - **Response cache** — TokenTriage's own store of answers it has seen before. A hit is served locally, so there's no provider call and no new tokens. [Cache accounting](/docs/concepts/cache-accounting) - **Price catalog** — the price table compiled into the binary: 2,889 priced model keys, each keyed `/`. That is a statistic about a vendored price table, not a claim about what TokenTriage can reach. Costing a request needs no network call. - **Pricing snapshot** — the stamp naming which build of that table produced a figure. This build carries `litellm@2026-07-14#179affb`. - **Settlement** — the second ledger row an async provider's job gets, written when the provider reports what the job cost. The submit row is written at once and carries no dollar; a reader takes the settlement's figure in place of it rather than adding the two. [Reading a settled cost](/docs/build/providers-settlement) - **EST** — the mark on a figure TokenTriage worked out from token counts and its price table, rather than one the provider reported. - **Metacost** — what the measuring cost you. The semantic response cache makes billable embedding calls to decide whether two prompts match; metacost is that spend, reported apart from the traffic it measured. - **Reconciliation residual** — the part of a provider invoice your ledger rows don't account for, shown as "unexplained" rather than folded into the matched total. - **FOCUS** — the FinOps industry standard for cost-export columns, so an export lands in the same tools as your cloud bills. - **Chargeback and showback** — two ways to split spend by team, tenant or feature. Chargeback moves money in your accounting; showback only reports who spent it. ## Routing and evidence - **Counterfactual** — a figure for a call that didn't happen. If TokenTriage says a cheaper tier would have cost $0.0004, that's a counterfactual: the same request repriced against a model it never saw. - **Savings verdicts** — how much weight a savings figure carries. `PROVEN` was measured after a change went live, `ESTIMATED` was repriced from a model, `UNPROVABLE` means the evidence isn't there. A recommendation you haven't acted on can't reach `PROVEN`. - **Honesty verdict** — the answer `eval` gives to "is routing earning its keep?", in one of four words printed as the API returns them: `BEAT`, `NOT_BEAT`, `ON_FRONTIER`, `CANNOT_MEASURE`. Grading only served answers can place routing *on* the frontier; a *beat* takes shadow duplication. - **Zero-router baseline** — the best score reachable with no per-request decision, by splitting traffic between your models in fixed proportions set in advance. It's the bar `eval` measures against. - **Shadow duplication** — sending the same query to both tiers and grading both answers. It's the only way to see what the other tier would have said, and the only evidence that a router beats the baseline. - **Calibration** — fitting your routing thresholds to your own decision log rather than a guessed number. It writes a dated candidate file that changes nothing until you activate it. [Calibration and evals](/docs/calibration/overview) - **Cache affinity** — a short-lived pin keeping later turns of one trace on the tier that warmed the provider's prompt cache, so a warm prefix isn't discarded. ## LLM judges - **LLM judge** — a model asked to grade another model's output, so quality gets a score without a person reading every row. Each graded request costs one extra model call. [LLM judges](/docs/judges/overview) - **Judge subject** — what a validation result is *about*: one judge, on one metric, in one segment, at one version of itself. Six coordinates in all, three that file the result and three that pin it to a version. [How a judge reaches a measured basis](/docs/judges/agreement) - **Judge method** — the shape of question a judge is asked. Five are registered: `pointwise-bool` (pass or fail on one answer), `pointwise-rubric` (a score against written criteria), `pairwise` (which of two answers is better), `reference` (how close an answer is to a known-good one, also called reference-based), and `panel` (several judges vote). Two of the five compose from a config file; the other three are turned away at `validate` and again at boot. [Configure a judge](/docs/judges/configure-a-judge) - **Pointwise and pairwise** — the two shapes of judging question. Pointwise grades one answer on its own; pairwise shows a judge two answers and asks which is better, which is why only pairwise has a presentation order to swap. - **Rubric** — the written criteria a rubric judge scores an answer against, named by an id in config rather than pasted into it. - **Panel** — one judge made of several seats drawn from more than one model family, all voting on the same item. Configuration enforces a floor of three seats and sets no ceiling, and an odd panel always decides. [Run a cross-family panel](/docs/judges/run-a-panel) - **Same-family refusal** — the rule that turns away a judging setup whose grader belongs to the same model family as a tier it would grade, so no model marks its own family's homework. [Position bias, self-preference and panels](/docs/judges/bias-controls) - **Both-orderings trial** — running a pairwise comparison twice with the two answers presented in each order, so a judge that prefers whichever it reads first is caught rather than believed. - **Swap flip** — the share of both-orderings trials whose two runs named different winners. It has a ceiling of its own, `0.10`, and a flip rate of `0.0` is a measurement rather than a missing value. - **Judge-uncertain** — the name for a pair whose two orderings disagreed. It is counted and left out of the arm's estimate rather than averaged in. On the wire it arrives as a `flipped` count, and as an abstention reason it is spelled `swap_inconsistent`. [Judge vocabularies](/docs/reference/judge-vocabularies) - **Gold set** — the human-labelled rows a judge is measured against, assembled from your own ledger. Rows carrying demo, synthetic, unattributed or superseded provenance are excluded and counted before anything is measured. [Collect the labels a judge is measured against](/docs/judges/collect-labels) - **Gold-set revision** — a short digest naming which version of a gold set a validation run was measured against, so two runs can be told apart. It is empty wherever the record it came from has none. - **Label event** — one `(annotator, request)` pair that survived deduplication and carries a judge verdict to pair against. It is the unit the sample-size floor counts, which is why one gold set can hold 105 label events over 80 requests. - **Overlap sample** — the requests two annotators both labelled. Agreement between the people is measured on these rows and nowhere else, so a queue that hands every row to one person produces no overlap and no agreement figure. - **Inter-annotator agreement** — how far two people labelling the same rows agree, measured over the overlap sample. It gates the gold set: where the people agree too little, the judge is not measured against them at all. - **Cohen's kappa** — agreement between two labellers, corrected for the agreement chance alone would produce. A kappa of `0.0` means chance level, which is a finding rather than a missing number, and it travels as a point estimate beside the count it was computed over. - **Promotion** — the record that a judge subject cleared every floor, filed under its judge id, metric and segment, and pinned to its model, prompt digest and evaluator version. It is what licenses a `measured` quality basis. [Validate a judge and promote it](/docs/judges/validate-and-promote) - **Demotion** — a promotion withdrawn. Re-running the validation check against a subject whose promotion still stands is what withdraws one, and the subject's timeline keeps a `demoted` entry saying so. [Judge drift, with no monitor between checks](/docs/judges/drift) - **Method version** — the digest identifying which judge method and which prompt version a reading was taken under. A prompt-regression reading means nothing without it. - **Abstention** — a verdict whose content is that there is no verdict, carrying one of four named reasons. It lands in a counted bucket of its own rather than in a pass rate, and it cannot also carry a score. [When a judge can't tell](/docs/judges/abstention) - **Selective risk** — a certification about the answers a judge is confident enough to give: how often it is wrong inside that accept region, bounded on held-out data. Where the process that answered holds no fit, the reading comes back null rather than zero. - **Clopper–Pearson lower bound** — the exact one-sided bound the licensing arithmetic prints as `CP-LCB`, computed by `clopper-pearson-v1`. It is the number a floor is compared against, not the estimate printed beside it. - **Benchmark agreement** — a judge's agreement with a public meta-evaluation corpus. It is a pre-screen and a prompt-regression check, and it licenses no promotion, no ceiling and no `measured` basis, because leaderboard validity is not your-traffic validity. - **Annotation queue** — the surface that offers unlabelled rows to a person to grade, ranked by a strategy you choose. Labels come back through the outcomes endpoint rather than through a second door of its own. [Collect the labels a judge is measured against](/docs/judges/collect-labels) - **Lease** — the hold a claim puts on the rows it offered, so two annotators are not handed the same request. Submitting a label does not release it; only expiry does. - **Quality basis** — what kind of claim a quality number is: `measured`, `reported`, or `insufficient`. It is a different enum from the guardrail basis below, and the two spell `reported` with different meanings. [Why your number is reported, not measured](/docs/judges/basis-ceilings) ## Governance and keys - **Virtual key** — a bearer token you mint for a team or application. It starts with `ttv_`, attaches to one place in your org chart, is stored as a SHA-256 hash, and is shown in full once, at mint. [Governance](/docs/build/governance) - **Tenancy node** — a position in the `org / team / user / key` hierarchy. Limits and keys attach to a node; spend rolls up the tree. - **The six limit types** — `budget_usd`, `quota_tokens`, `quota_requests`, `rpm`, `tpm`, and `concurrency`. Everything you can cap is one of these. - **Limit mode** — whether one limit is `off`, `shadow` (records what it would have stopped) or `on` (stops it). Governance stays off until you enable it, but a limit you write defaults to `on`, so start new ones in `shadow`. - **Row == wire** — the response your caller gets and the ledger line are built from the same value before any byte goes out, so they can't disagree about why a request was turned away. - **Fail-open with disclosure** — when the counter behind a limit can't be reached, the limit is suspended. Traffic keeps flowing and the row records `suspended:[]`, so you get a working request plus a note that the cap went unchecked. ## Guardrails - **Binding** — one configured guardrail: its category, its detectors, where in the request they run, and what happens on a finding. Its `id` is what the row's `engaged[]`, `fired[]` and `suspended[]` carry. [Guardrails overview](/docs/guardrails/overview) - **Detector** — the code that reads text and reports a finding, such as a credit-card matcher. 21 are registered in this build. - **Category** — what a binding looks for: `pii`, `secrets`, `injection`, and so on. The vocabulary is a fixed set of nine, seven with a detector here. It says what the ledger can record, not what the binary can find. - **Phase** — when a check runs: `input` before the provider sees it, `output` on a complete reply, or `stream` on a reply arriving in pieces. [What gets inspected](/docs/guardrails/what-gets-inspected) - **Locality** — where a detector runs. `embedded` is in-process with no network access, `judge` is an LLM call, `sidecar` is an HTTP classifier you host. The last two need an explicit consent string in config before the proxy starts. [Air gap and limits](/docs/guardrails/air-gap-and-limits) - **Basis (guardrail)** — where a confidence number came from, and so how much to trust it. `exact`: a deterministic check passed. `fixture`: this detector version's measured precision. `calibrated`: a calibration file licenses it, and it alone may carry a range. `reported`: the detector's raw score. `insufficient`: too little evidence for any of those. Five values, and a different enum from the quality basis above, which has three and spells `reported` to mean something else. [Guardrail confidence](/docs/guardrails/confidence) - **Effect** — what happened to the bytes: `mutated`, `blocked`, `annotated`, or `warned`. That's a separate fact from what the binding asked for, since redacting something absent changes nothing. - **Suspension** — a check that couldn't run, recorded on the row with a reason from a fixed list such as `detector_timeout`. It tells a clean row from an unguarded one. ## Observability and analytics - **Business dimensions** — the labels you attach to spend so it can be sliced later: `dim.user`, `dim.tenant`, `dim.feature`, `dim.tag.`. They come from your `x-tt-*` headers. - **Rollup tuple** — the eight columns cost analytics pre-totals by, once an hour, so a dashboard query doesn't rescan the ledger: `tenant`, `team`, `key`, `feature`, `model_served`, `tier`, `provider`, `endpoint`. [Cost analytics](/docs/build/cost-analytics) - **BubbleUp** — the ranking that answers "which slice moved?". When spend jumps, it sorts your dimensions by how much each contributed. - **`robust-seasonal-v1`** — the method behind cost anomaly detection, printed beside every finding so you can tell what produced it. [Anomaly detection](/docs/build/anomaly-detection) - **Coverage merge** — how a multi-replica deployment answers without a shared log. Each replica answers from its own ledger, the results are combined, and the total says how many replicas it covers. [Scale out](/docs/operate/scale-out) ## See also - [Core concepts](/docs/concepts/core-concepts) — the trust ladder and decision ledger in context. - [How TokenTriage works](/docs/concepts/how-it-works) — one request followed end to end. - [The honesty model](/docs/concepts/honesty-model) — provenance, UNPRICED, and what each verdict claims. - [Decision ledger fields](/docs/reference/ledger-fields) — every field on a ledger row, with types.