# The honesty model
How TokenTriage separates missing prices from zero cost, measured savings from estimates, and routing wins from results it cannot judge.
Source: https://tokentriage.com/docs/concepts/honesty-model
---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
import RoutingVerdictDiagram from '../../../components/RoutingVerdictDiagram.astro';
A cost total is useful only if you know what it leaves out. A routing result is useful only
if it was compared with a meaningful baseline.
TokenTriage keeps those limits attached to the number. Missing prices remain unpriced instead
of becoming `$0`. Savings say whether they were measured or estimated. Routing gets a win only
when both choices were scored on the same requests.
- **Checking a cost or savings figure?** Start with
[Provenance & UNPRICED](#provenance--unpriced).
- **Evaluating routing?** Skip to
[why routing needs a baseline](#why-routing-beat-the-baseline-is-hard-to-show).
[How TokenTriage works](/docs/concepts/how-it-works) follows a request end to end. If
`ledger`, `shadow`, and `route` are new to you, read the
[trust ladder](/docs/concepts/core-concepts#the-trust-ladder) first.
## The invariants
Two rules explain the rest of this page:
1. **Unknown stays visible.** A missing price does not become zero, and missing evidence does
not become a claim.
2. **The evidence limits the conclusion.** A forecast remains an estimate. A routing win
requires a comparison that can show the router made a better choice.
Those rules also shape how TokenTriage behaves outside cost and routing.
- Costing, analytics, and the dashboard run off the request path, so bookkeeping does not delay
your traffic.
- On an internal error, the proxy falls back to verbatim passthrough, including streaming.
- Request attribution comes from the `x-tt-*` headers your client sends. TokenTriage does not
infer ownership from timing or prompt text.
- The JSONL [decision ledger](/docs/concepts/core-concepts#the-decision-ledger) is the record.
The SQLite index and dashboard views can be rebuilt from it.
- A guard that could not run records why in `suspended[]`; it does not look like a clean row.
Security and administrative guarantees live in
[security](/docs/operate/security) and [governance](/docs/build/governance).
## Provenance & UNPRICED
A cost figure needs enough context to tell you whether it is complete. TokenTriage keeps three
facts beside each amount:
- which price-list version was used;
- whether token counts were measured or estimated; and
- whether TokenTriage found a price for the model.
An estimated figure carries an `EST` flag. A model with no matching price follows a different
rule.
**UNPRICED is not zero.** If TokenTriage cannot find a rate, the row has a `null` cost and is
excluded from the arithmetic. The total states how many rows were excluded, for example:
`$41.20 — COST excludes 116 unpriced rows of 2,020`.
That `$41.20` is the known spend for priced rows, not a complete total. The 116 excluded rows
are not treated as free.
A model you declared free, such as a local endpoint, is different. It gets an explicit `$0`,
`unpriced: false`, and the price-list version that supplied the zero rate. This keeps “we spent
nothing” separate from “we do not know what we spent.”
The same rule applies during reconciliation. If the provider invoice and ledger do not match,
the remaining amount is shown as **unexplained** instead of being changed to zero. See
[provider-bill reconciliation](/docs/build/cost-reconciliation).
A cost is an amount in USD plus the `estimated` and `unpriced` flags. Its ledger record also
carries `pricing_snapshot`, which identifies the price list used for the calculation.
[Ledger fields](/docs/reference/ledger-fields) defines each field.
## The savings verdicts
A savings figure also tells you what kind of claim it makes. TokenTriage uses three labels:
| Verdict | Plain meaning | Typical example |
|---|---|---|
| `PROVEN` | measured from events that have already happened | savings measured after a change, or avoided spend measured by replay |
| `ESTIMATED` | calculated from recorded usage and stated assumptions | a recommendation about future traffic |
| `UNPROVABLE` | the available evidence cannot support a dollar figure | a slice with missing or insufficient evidence |
These labels describe the source of the figure, not how confident the UI feels about it. An
`UNPROVABLE` result renders no dollar and names the missing evidence.
**A recommendation is `ESTIMATED`, not `PROVEN`.** It forecasts traffic that has not happened
yet. `PROVEN` is reserved for a post-change or replay measurement.
Only two method IDs can produce a `PROVEN` figure. The
[cost figure reference](/docs/build/cost-figure-reference#the-methods) lists
them and the evidence each one requires.
## Two verdict vocabularies — don't conflate them
Savings and routing use different verdicts because they answer different questions.
| | **Savings / impact verdict** | **Routing verdict** |
|---|---|---|
| Question | Is this dollar figure measured, estimated, or unavailable? | Did routing beat a blind model mix at the same cost? |
| Values | `PROVEN` · `ESTIMATED` · `UNPROVABLE` | `BEAT` · `NOT_BEAT` · `ON_FRONTIER` · `CANNOT_MEASURE` |
| Where it appears | recommendations, measured savings, and reconciliation | the routing evidence panel and `eval` output |
| Missing evidence | `UNPROVABLE`; no dollar is shown | `CANNOT_MEASURE`; no routing judgment is made |
The verdicts can differ without contradicting each other. A recommendation may have strong
quality evidence and still show `ESTIMATED` savings because future traffic has not happened.
When you quote a verdict, say whether it describes savings or routing.
Quality labels and evidence bases are a separate vocabulary. They are collected in
[Calibration & evals vocabularies](/docs/reference/quality-vocabularies).
## Why "routing beat the baseline" is hard to show
Suppose you use a cheap model for 70% of requests and an expensive model for 30%, without
looking at the request first. That blind mix has an average cost and quality. Change the mix
and you trace a line between the two models.
TokenTriage calls the best line available from blind mixes the **zero-router frontier**. A router
must land above that line to show it chose well for individual requests. Landing on the line is
no better than choosing the same proportions at random.
The line in the diagram is what a blind mix can achieve. A result above it has better quality at
the same cost. A point on or below it does not show routing skill.
Your normal logs record the model the router chose and how that answer scored. They do not record
how another model would have answered the same request. With only those **served-model labels**, a
good choice and a lucky mix look the same.
**Judged shadow duplication** supplies the missing comparison. It sends the same request to both
models and grades both answers, while your application still receives the primary response. The
second graded answer is the counterfactual: what would have happened under the other choice.
This page does not cover how a model grader becomes trusted measurement. That process is explained
in [How a judge reaches a measured basis](/docs/judges/agreement) and
[Why your number is reported, not measured](/docs/judges/basis-ceilings).
## The honesty verdict for routing
`eval` turns the available routing evidence into one of four verdicts. The CLI, API, and dashboard
use the same names.
| Verdict | What it means | What to do next |
|---|---|---|
| `BEAT` | both choices were graded on the same requests, enough labels were available, and routing cleared the improvement threshold | if the evidence is fresh and matches the current policy, promote it to `route` |
| `NOT_BEAT` | the paired comparison ran with enough labels, but routing did not clear the threshold | keep `shadow` mode and adjust the policy |
| `ON_FRONTIER` | only served-model labels were available, so the result cannot show routing skill | enable judged shadow duplication |
| `CANNOT_MEASURE` | too few rows were labeled, or the filter matched no rows | collect labels; use judged shadow duplication if you need a routing-skill verdict |
Despite its name, `ON_FRONTIER` also covers a point below the frontier. Served-model labels can
place the result on or below the line, but they cannot show that per-request choices caused either
position.
With served-model labels and a point on the frontier:
```text
verdict: routing sits ON the observed-model frontier (Δ +0.0000) — served-model labels
(quality source 'oracle') score only the tier that served each row, so routing cannot be
shown to beat a blind model mix. Enable shadow duplication (shadow.duplicate) for a
counterfactual routing-skill verdict; use calibrate to sweep thresholds.
```
The same wire verdict prints `BELOW` when the point falls under the line:
```text
verdict: routing sits BELOW the observed-model frontier (Δ -0.0300) — served-model labels
(quality source 'oracle') score only the tier that served each row, so routing cannot be
shown to beat a blind model mix. Enable shadow duplication (shadow.duplicate) for a
counterfactual routing-skill verdict; use calibrate to sweep thresholds.
```
With a judged shadow comparison that cleared the threshold:
```text
verdict: routing BEATS zero-router by 0.2500 (counterfactual: judged shadow duplication)
```
With a judged shadow comparison that did not clear it:
```text
verdict: routing does NOT beat zero-router (Δ +0.0000) — keep shadow mode
```
With too few labels:
```text
verdict: insufficient labels to judge routing — enable oracle sampling (classifier.oracle)
or shadow duplication (shadow.duplicate)
```
A filter that matches zero rows also produces `CANNOT_MEASURE`; the evidence API reports
`matched_rows: 0` beside it. Promotion checks two more conditions after `BEAT`: the labels must
be fresh enough, and the policy fingerprint must match the policy being promoted. The fingerprint
identifies the policy version under which the evidence was collected.
## What this page doesn't cover
[Ledger fields](/docs/reference/ledger-fields) defines the stored records. The
[trust ladder](/docs/concepts/core-concepts#the-trust-ladder) explains `ledger`, `shadow`, and
`route`. [Cache accounting](/docs/concepts/cache-accounting) explains why provider prompt-cache
discounts and TokenTriage response-cache savings remain separate.