# Preview a routing policy change

Re-run past decisions under a draft policy and see which routes and estimated costs would change before you ship it.

Source: https://tokentriage.com/docs/build/routing-preview

---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';

Use `policy preview` before replacing a routing policy. It re-evaluates requests already in your
decision log with the draft, then reports which decisions would change and the estimated cost
difference.

The command runs offline. It reads the draft and the log, writes nothing, and sends no provider
requests.

## What you need

Prepare the draft and choose the traffic history you want to test it against.

| You need | Notes |
|---|---|
| a draft config | `--config` is required; the file is validated before the preview runs |
| a decision log with completed rows | the default is `~/.tokentriage/decisions.jsonl` |
| enough representative traffic | the result describes only the rows in this log |

You do not need to load the draft into a running proxy. The draft can stay on your workstation
while you preview a copied production log.

## Run the preview

Pass the draft config and, when needed, the log path:

```console
$ tokentriage policy preview --config ./draft.yaml --log ~/.tokentriage/decisions.jsonl
decision/cost SIMULATION (not a quality claim) — approximations: fixed_usage, recorded_runtime_state
rows:       198   changed: 157   would_cost_delta: $6.361757
provenance: 131 estimated-usage rows, 4 unpriced rows (excluded from dollars)
by rule (draft):
  default                  rows=132    changed=107    Δ$4.231154
  pin-tool-loops           rows=66     changed=50     Δ$2.130603
changed examples (req_ids): [tt_00000001 tt_00000002 tt_00000003]
pricing:    litellm@2026-07-14#179affb
```

Read that sample as follows: the draft was evaluated against 198 rows and would change 157 of
their routing decisions. Its positive cost delta means the draft is estimated to cost $6.36 more
over those replayed rows. A negative delta would mean an estimated saving.

The estimate excludes the four unpriced rows. It uses the original token usage for the other
rows, including the 131 whose usage was estimated rather than reported by a provider.

## Read the summary in order

Start with the whole-log result, then use the breakdown to find the cause.

1. **Check `rows`.** This is how many log rows the preview considered after applying its row cap.
2. **Check `changed`.** This counts rows whose draft decision differs from the logged decision and
   whose required inputs could be reconstructed. Cache-affinity pins and rows with unavailable
   inputs do not enter this count.
3. **Interpret `would_cost_delta`.** The calculation is draft cost minus logged cost. A positive
   value is additional estimated spend; a negative value is estimated savings.
4. **Read `provenance`.** Estimated-usage and unpriced counts tell you how much of the result rests
   on provider-reported usage and how much was excluded from dollars.
5. **Use `by rule` and the request ids.** The rule rows show where the changes came from. Explain
   one of the example request ids when you need the inputs behind an individual change.

The delta covers the replayed rows as a group. It is not a per-request figure and it does not
project future traffic volume.

## Understand the two approximations

The first line names the assumptions that make an offline preview possible.

| Label | What the preview does | What it does not claim |
|---|---|---|
| `fixed_usage` | prices the draft tier with the token counts recorded on the original row | that a different model would produce the same number of tokens |
| `recorded_runtime_state` | rebuilds runtime inputs from facts stored on the row | that budgets, latency, quality, or other state still have those values now |

<Callout type="warning">

This is a routing and cost simulation, not a quality result. A cheaper draft may still produce
worse answers. Use shadow traffic and evaluation results before promoting a policy whose quality
risk matters.

</Callout>

## Know which rows are excluded

Exclusions are kept out of the calculation they cannot support instead of being treated as
unchanged or free.

| Row type | Treatment | Why |
|---|---|---|
| cache-affinity pinned | excluded from `changed` and the cost delta | runtime cache state, not the rule engine, chose its tier |
| unavailable draft input | included in `rows`, but excluded from `changed` and the cost delta | the log does not contain enough state to re-evaluate the draft confidently |
| unpriced | excluded from the dollar delta | either the logged route or draft route has no usable price |

When a draft reads runtime inputs a row did not record, the command prints a `DISCLOSURE` line
with the row count and input names. If a strict majority of rows lack required inputs, the preview
stops instead of reporting an aggregate over mostly incomplete re-evaluations.

<Details summary="Row-cap and demo-data limits">

The command keeps the newest 5,000 rows. The cap is not adjustable from the command line, and the
output notes when older rows were dropped.

Synthetic demo rows count like other rows. The CLI does not separate them in this summary, so use
a log whose origin you know when the result will inform a production rollout.

</Details>

## Fix common problems

Use the message or surprising number to decide what to inspect next.

| Symptom | What it means | What to do |
|---|---|---|
| the draft does not compile | the config is invalid or names a policy primitive this build cannot use | validate the config and follow the path in the error |
| a `DISCLOSURE` line appears | some rows do not contain runtime inputs the draft needs | inspect the named inputs; collect new shadow rows if they are important to the policy |
| the preview stops as mostly unevaluable | more than half of the rows lack required inputs | use a newer log or simplify the draft's runtime dependencies |
| the cost delta looks unexpectedly small | pinned, unavailable, or unpriced rows were excluded | compare the exclusion counts with `rows` before trusting the dollar total |
| many decisions change unexpectedly | one draft rule is catching more traffic than intended | start with the largest `by rule` count, then explain one of its example request ids |

`POST /tt/api/v1/replay` provides the same kind of preview for automation. Its response keeps the
approximation labels and uses a typed money value so an all-unpriced result is not reported as a
zero-dollar change.

<NextSteps items={[
  { title: 'Explain a routing decision', href: '/docs/build/routing-explain', description: 'Inspect one changed request and see the rule, path, and values behind it.' },
  { title: 'Turn routing on', href: '/docs/build/routing', description: 'Move a reviewed policy from ledger to shadow and then to enforced routing.' },
  { title: 'Routing explanation fields', href: '/docs/build/routing-explanation-fields', description: 'Look up the selector, draw, and cascade details recorded on each decision.' },
  { title: 'CLI reference', href: '/docs/reference/cli', description: 'Check the complete flags for policy preview and the other routing commands.' }
]} />