# Gate a rollout on a validated judge
Turn a validated judge into a routing decision — set a quality floor, reload, and watch a candidate tier be admitted or turned away.
Source: https://tokentriage.com/docs/judges/gate-a-rollout
---
import Callout from '../../../components/Callout.astro';
import NextSteps from '../../../components/NextSteps.astro';
import EvalProgramDiagram from '../../../components/EvalProgramDiagram.astro';
import QualityRoutingSeamDiagram from '../../../components/QualityRoutingSeamDiagram.astro';
A cheaper model would save you money on half your traffic, and you would like to send it there
only while somebody can show the answers are still good enough. You have the grading in place and
a promotion behind it. What is left is the step that turns that record into an admission rule the
router applies to every request.
That step is a floor on a selector. A candidate tier becomes eligible only while its lower
confidence bound clears the number you set, and a tier that cannot clear it stops being an option
rather than becoming a cheap one. This page sets the floor, reloads, and reads the decision back.
Two pages come first and this one assumes both. Getting a subject through the validation walk is
[Validate a judge and promote it](/docs/judges/validate-and-promote); what a promotion moves and
what it leaves alone is
[Why your number is reported, not measured](/docs/judges/basis-ceilings). The floor's own grammar,
its selector parameters and the score-table terms it writes are
[Gate routing on a proven quality floor](/docs/calibration/gate-routing-on-quality), which covers
the same mechanism for every metric rather than for a judged one.
## What has to be true before the floor admits anything
Four conditions, and the first two are the judge half. Check them in this order, because each one
makes the next readable.
| Condition | How you check it | If it does not hold |
|---|---|---|
| A promotion stands for the subject | `tokentriage evals validate` prints `reading: state=promoted` | the walk declined; read the floor it named |
| The rows you want to gate on were written after the promotion instant | compare the promotion time against your window | re-read a later window |
| An `active` quality calibration covers the segment | `tokentriage quality reliability` | fit one, then activate it |
| The selector is a cost-aware one | `cheapest_meeting`, `epsilon_greedy` or `weighted_objective` | the floor is a parameter of those three |
What you do **not** need: a running server for the check itself, a judge credential to read the
record back, or any change to the tiers themselves. The floor is added to a `select` node and
nothing else moves.
The floor is compared against a **calibrated lower bound**, not against a judge's score. A
promotion lifts the ceiling so that judged evidence may be treated as a measurement; the
calibration is what turns those measurements into a bound with an error somebody checked. Both
have to be there, and they are two different artefacts with two different lifecycles.
## 1. Confirm the promotion stands, and note when it landed
Run the validation verb and read one line off the record for your subject.
```console
$ tokentriage evals validate --config ./tokentriage.yaml --state-dir ./state/dash
```
The line that answers this question is `reading: state=promoted`, printed under the subject's
block beside `gold_revision` and `subjects_observed`. A subject that has been through a run that
declined reads `never_validated` instead, and the `refusal` lines above it name the floor with
what it wanted beside what it had. [Validate a judge and promote it](/docs/judges/validate-and-promote)
reads a whole record of each kind.
Write down when the promotion landed. A promotion is forward-only: the quality basis on a row is
derived at read time from the row's own timestamp, so every judged row written before that instant
keeps its `reported` basis permanently. A window that ends before the promotion licenses nothing,
however well the judge validated.
## 2. Fit a calibration on the segment you want to gate
The floor reads `candidate.quality.lcb`, which comes off an `active` calibration artifact for that
segment. Without one there is no bound to compare, and the selector's refusal says so in those
words when you try to set a metric on its own.
Fit and activate as you would for any metric, over a window that **starts after** the promotion
instant from step 1. [Calibrate quality from your logs](/docs/calibration/calibrate-from-logs) is
the procedure; the only judge-specific part is the window.
Offline proof and the online gate are two different jobs, and it is worth seeing where each one
sits before you wire the second.
Read that left to right. A dataset of ledger references drives one instrument; the fold keeps arms
of different bases apart instead of blending them; only priced points reach the frontier; and the
answer is one of four states. Of the three instruments, only the counterfactual one can reach
`BEAT`, so an offline run tells you where a tier sits and a shadow dual-run tells you it would have
won. That is the evidence you bring to the floor. The floor itself is the next diagram, and it runs
on every request rather than on a dataset.
## 3. Put the floor on the selector
`min_quality_lcb` and `quality_metric` live together in a `select` node's `by.params`. This admits
the cheapest tier whose bound clears `0.90` on `task_completion`, and sends everything else to the
frontier when none qualifies.
```yaml
routing:
default_tier: frontier
policy:
version: 1
root:
rules:
- id: task-completion-floor
to:
select:
from: { tiers: all }
by:
selector: cheapest_meeting
params:
min_quality_lcb: 0.90
quality_metric: task_completion
on_empty: { tier: frontier }
default: { tier: frontier }
```
`task_completion` is the metric `judge-task-completion` scores, so the floor and the judge are
talking about the same property. The `on_empty` target is where traffic goes when no candidate
clears the floor, and it is a routing decision the ledger records rather than a silent fallback.
## 4. Validate, then reload
Validate the whole file before anything reloads. The block above was loaded by the binary as part
of the config this page was written against.
```console
$ tokentriage validate --config tokentriage.yaml
mode: ledger
listen: 127.0.0.1:8787 admin: 127.0.0.1:9090
upstreams: 1
- openai https://api.openai.com
tiers: 2 (cheapest -> most capable)
0. cheap -> openai/gpt-5-mini
1. frontier -> openai/gpt-5
rules: 0
OK: configuration is valid
```
Exit status is 0. That capture is from a source build on Windows, run from the config's own
directory so no path appears; two egress warnings for the judge evaluator were printed above it on
stdout, twice each, and are elided here. Two lines are worth pre-empting because they read like faults and are
not. `upstreams: 1` counts upstreams and not tiers, and **`rules: 0` counts the legacy
`routing.rules` list rather than the policy tree** — the shipped
`examples/evals/journey-3-hard-floor-airgapped.yaml`, whose whole subject is a floor, prints the
same `0`.
The router reads calibration off an immutable snapshot published away from the request path, so
the floor takes effect on the next reload rather than part-way through a request in flight.
Above the divider, off the request path, the active artifact is folded in the background and
published as a snapshot. Below it, on the request path, each candidate tier does one map probe of
that snapshot, and the gate admits a tier whose lower bound clears the floor and excludes one whose
bound sits below it. The dashed connector crossing the divider is the only traffic between the two
halves, which is why a reload rather than a request is what moves the floor.
## 5. Read a decision back and find the term that explains it
Send traffic, then read one decision. The selector's score table carries every candidate's quality
verdict rather than only the admitted ones.
```console
$ tokentriage policy explain --req-id --log ./state/decisions.jsonl
```
Three outcomes are possible for a candidate, and they are three different facts.
| What you see on the score table | What happened | What to change |
|---|---|---|
| the tier was chosen, no quality term | its bound cleared the floor | nothing |
| `quality_below_min_lcb` | the tier was measured, admitted to the comparison, and its bound fell short | lower the floor, or improve the tier |
| `quality_absent:` | the tier's quality could not be read at all, and the reason rides the term name | the reason names it: fit a calibration, wait for evidence, or promote the judge |
The distinction in the last two rows is the one worth holding on to. Absence is disclosure, not a
penalty: a tier nobody could measure is excluded and disclosed rather than scored as having failed.
## Three ways the floor admits nothing, and how to tell them apart
Each of these ends with the same observation — traffic on the `on_empty` target and a cheaper tier
taking none of it. They have three different causes and three different fixes.
**The judged evidence is capped, so the segment has no measured figure.** Ask the quality surface
for a measured figure on that segment and it answers `basis_ceiling_reported`, with `need` and
`have` counts and a hint naming the remedy. A judge signal is promoted only by the validation
protocol, per judge tuple per metric per segment, and no promotion stands for these rows' subject.
The same hint says the surface **cannot tell you which of three reasons applies**. The registry
holds promotions only, so an unvalidated judge, a declined one and a demoted one all read alike
there. `tokentriage evals validate` serves the record with its named floor and its need and have.
**The promotion is for a different segment.** A promotion is filed under one judge id, metric and
segment, and pinned to the judge model, the prompt sha and the evaluator version. The same judge
on another tier is another subject and an unvalidated one, so a floor on the second segment admits
nothing while the first is fine.
**The metric id is misspelled.** This one clears every check you would think to run.
```console
$ tokentriage validate --config typo.yaml
…
rules: 0
OK: configuration is valid
```
Exit status 0, on a file whose `quality_metric` reads `task_completeion`. The id is not checked
against any registry at validate time, so the mistake surfaces at runtime as a metric that joins
no segment: every candidate's quality fact is absent under a closed token, every candidate is
excluded and disclosed, and traffic lands on `on_empty`. Check the id against the evaluator's
declared metrics by hand, and read the score table after the first request rather than trusting a
green validate.
Two neighbouring mistakes in the same block are caught, which is what makes the third one
surprising. `quality_metric` set with no `min_quality_lcb` is turned away at validate with
*"quality_metric is set without min_quality_lcb, so no calibrated bound would be read"*, and a
floor outside `[0,1]` with *"min_quality_lcb must be in [0,1] (it is a bound on a pass rate), got
1.4"*. Both print twice, because this binary writes every error once through its command framework
and once from `main`.
## Keep the floor current after it is live
A promotion is a measurement of a judge against labels collected at one moment, and three things
under it can move afterwards: the grading model, the prompt it is sent, and the mix of requests it
grades. The first two reset the promotion by themselves, because the pin stops matching and the
subject reads `version_changed`. The third moves nothing you can see.
So re-run the validation check on a cadence you pick, and treat a prompt edit as a re-validation.
A run that falls under the floor demotes at the same version; a run that clears it re-promotes and
re-opens the window.
[Judge drift, with no monitor between checks](/docs/judges/drift) has the full account of which
part of that loop you own.
## What this page does not cover
**The full selector grammar.** The three cost-aware selectors, where a `select` node sits in a
policy tree, and why naming `candidate.quality.mean` is a compile error are
[the policy language](/docs/build/policy-language) and
[Gate routing on a proven quality floor](/docs/calibration/gate-routing-on-quality).
**Fitting and activating a calibration.** The B.5 gate, the holdout split and the reliability
report belong to the calibration track.
[How calibration works](/docs/calibration/how-calibration-works) is the engine and
[Calibrate quality from your logs](/docs/calibration/calibrate-from-logs) is the procedure.
**Why a figure reads `reported` when a promotion did land.** A quality basis is the lowest of
several caps, and only the judge-tied ones yield to a promotion.
[Why your number is reported, not measured](/docs/judges/basis-ceilings) finds the one that binds.
**Every other symptom.** A floor that admits nothing is one observation among several.
[Why didn't my judge validate?](/docs/judges/troubleshooting) indexes the rest by what you can see.