Preview a routing policy change

Use policy preview before replacing a routing policy. It re-evaluates requests already in your decision log with the draft, then reports which decisions would change and the estimated cost difference.

The command runs offline. It reads the draft and the log, writes nothing, and sends no provider requests.

What you need

Prepare the draft and choose the traffic history you want to test it against.

You need	Notes
a draft config	--config is required; the file is validated before the preview runs
a decision log with completed rows	the default is ~/.tokentriage/decisions.jsonl
enough representative traffic	the result describes only the rows in this log

You do not need to load the draft into a running proxy. The draft can stay on your workstation while you preview a copied production log.

Run the preview

Pass the draft config and, when needed, the log path:

$ tokentriage policy preview --config ./draft.yaml --log ~/.tokentriage/decisions.jsonl
decision/cost SIMULATION (not a quality claim) — approximations: fixed_usage, recorded_runtime_state
rows:       198   changed: 157   would_cost_delta: $6.361757
provenance: 131 estimated-usage rows, 4 unpriced rows (excluded from dollars)
by rule (draft):
  default                  rows=132    changed=107    Δ$4.231154
  pin-tool-loops           rows=66     changed=50     Δ$2.130603
changed examples (req_ids): [tt_00000001 tt_00000002 tt_00000003]
pricing:    litellm@2026-07-14#179affb

Read that sample as follows: the draft was evaluated against 198 rows and would change 157 of their routing decisions. Its positive cost delta means the draft is estimated to cost $6.36 more over those replayed rows. A negative delta would mean an estimated saving.

The estimate excludes the four unpriced rows. It uses the original token usage for the other rows, including the 131 whose usage was estimated rather than reported by a provider.

Read the summary in order

Start with the whole-log result, then use the breakdown to find the cause.

1. Check rows. This is how many log rows the preview considered after applying its row cap.
2. Check changed. This counts rows whose draft decision differs from the logged decision and whose required inputs could be reconstructed. Cache-affinity pins and rows with unavailable inputs do not enter this count.
3. Interpret would_cost_delta. The calculation is draft cost minus logged cost. A positive value is additional estimated spend; a negative value is estimated savings.
4. Read provenance. Estimated-usage and unpriced counts tell you how much of the result rests on provider-reported usage and how much was excluded from dollars.
5. Use by rule and the request ids. The rule rows show where the changes came from. Explain one of the example request ids when you need the inputs behind an individual change.

The delta covers the replayed rows as a group. It is not a per-request figure and it does not project future traffic volume.

Understand the two approximations

The first line names the assumptions that make an offline preview possible.

Label	What the preview does	What it does not claim
fixed_usage	prices the draft tier with the token counts recorded on the original row	that a different model would produce the same number of tokens
recorded_runtime_state	rebuilds runtime inputs from facts stored on the row	that budgets, latency, quality, or other state still have those values now

Know which rows are excluded

Exclusions are kept out of the calculation they cannot support instead of being treated as unchanged or free.

Row type	Treatment	Why
cache-affinity pinned	excluded from changed and the cost delta	runtime cache state, not the rule engine, chose its tier
unavailable draft input	included in rows, but excluded from changed and the cost delta	the log does not contain enough state to re-evaluate the draft confidently
unpriced	excluded from the dollar delta	either the logged route or draft route has no usable price

When a draft reads runtime inputs a row did not record, the command prints a DISCLOSURE line with the row count and input names. If a strict majority of rows lack required inputs, the preview stops instead of reporting an aggregate over mostly incomplete re-evaluations.

Row-cap and demo-data limits

The command keeps the newest 5,000 rows. The cap is not adjustable from the command line, and the output notes when older rows were dropped.

Synthetic demo rows count like other rows. The CLI does not separate them in this summary, so use a log whose origin you know when the result will inform a production rollout.

Fix common problems

Use the message or surprising number to decide what to inspect next.

Symptom	What it means	What to do
the draft does not compile	the config is invalid or names a policy primitive this build cannot use	validate the config and follow the path in the error
a DISCLOSURE line appears	some rows do not contain runtime inputs the draft needs	inspect the named inputs; collect new shadow rows if they are important to the policy
the preview stops as mostly unevaluable	more than half of the rows lack required inputs	use a newer log or simplify the draft’s runtime dependencies
the cost delta looks unexpectedly small	pinned, unavailable, or unpriced rows were excluded	compare the exclusion counts with rows before trusting the dollar total
many decisions change unexpectedly	one draft rule is catching more traffic than intended	start with the largest by rule count, then explain one of its example request ids

POST /tt/api/v1/replay provides the same kind of preview for automation. Its response keeps the approximation labels and uses a typed money value so an all-unpriced result is not reported as a zero-dollar change.