# Routing policy language Reference for writing and validating routing.policy YAML, with examples for rules, selection, traffic splits, cascades, conditions, and terminal targets. Source: https://tokentriage.com/docs/build/policy-language --- import Callout from "../../../components/Callout.astro"; import Details from "../../../components/Details.astro"; import NextSteps from "../../../components/NextSteps.astro"; import PolicyTreeDiagram from "../../../components/PolicyTreeDiagram.astro"; Use `routing.policy` to choose which configured tier handles each request. If this is your first policy, start with the [routing overview](/docs/build/routing-overview) or follow [your first routing rollout](/docs/get-started/first-routing-rollout). Use this page while writing or troubleshooting the YAML. Every policy starts at `root`. Four decision nodes — `rules`, `select`, `split`, and `cascade` — eventually produce a terminal target such as a tier, refusal, or passthrough. TokenTriage records an explanation each time it evaluates the policy. ## Start with a minimal policy This policy sends requests that use tools to the `frontier` tier. Every other request goes to `cheap`. ```yaml routing: policy: version: 1 root: rules: - id: tools-to-frontier when: "request.has_tools" to: { tier: frontier } default: { tier: cheap } ``` After adding a policy, validate the complete config before starting TokenTriage: ```console $ tokentriage validate --config tokentriage.yaml ``` ## The policy document A policy document supports eight top-level keys. Unknown keys fail validation, so a typo cannot silently turn into a setting that does nothing. | Key | Type | Required | Notes | | ----------------- | ------ | ------------- | ---------------------------------------------------------------------------- | | `version` | number | no | `1`, or omitted; validation rejects any other value | | `seed` | string | conditionally | required for `split`; recommended for selectors that make randomized choices | | `on_no_candidate` | string | no | `fail_safe` (default), `refuse`, or `default_tier` | | `scorers` | map | no | named signal providers; each scorer validates its own settings | | `params` | map | no | named numbers, referenced as `$name` | | `predicates` | map | no | named conditions, referenced as `@name` | | `pools` | map | no | named candidate groups | | `root` | node | **yes** | the decision tree | The expanded example below shows how the optional parameter, predicate, and pool blocks connect to the decision tree: ```yaml routing: policy: version: 1 seed: "ab-1" on_no_candidate: default_tier params: simple_cut: { value: 0.35, calibrate: { signal: complexity, direction: below } } predicates: is_simple: "signal.complexity < $simple_cut" pools: cheap_pool: { members: [{ tier: cheap }, { tier: cheap-b, weight: 2 }] } root: rules: - id: simple-load-balance when: "@is_simple" to: { pool: cheap_pool } default: { tier: frontier } ``` `routing.policy` is mutually exclusive with the older `routing.rules` list. Setting both fails validation with `routing: routing.rules and routing.policy are mutually exclusive; set exactly one (both present)`. `on_no_candidate` decides what happens when a `select` finds no eligible candidate and has no `on_empty` branch. The default, `fail_safe`, forwards the request without choosing a tier and records why. `refuse` returns a typed `503` with the reason `no_candidate_available`. `default_tier` routes to `routing.default_tier`. ## Decision nodes and terminal targets Choose the decision node that matches what the policy needs to do: - **`rules`** for straightforward if/else routing based on request data. - **`select`** to filter several tiers and choose the best eligible one. - **`split`** for reproducible A/B tests and canary traffic. - **`cascade`** to try an answer, grade it, and escalate when necessary. - **A terminal target** to finish with a tier, refusal, passthrough, or another terminal result. Each node contains exactly one of these forms. For example, do not put both `rules` and `select` on the same node. Validation points to the node when two forms are mixed. ### `rules` — first match wins Use rules for predictable routing based on information already available on the request. The first matching entry wins; `default` handles every request that did not match. The list and its `default` are sibling keys on the node, not nested under a `rules:` mapping. ```yaml rules: - id: pin-tool-loops when: 'trace.step_class = "tool_loop"' mode: route to: { tier: frontier } default: { tier: cheap } ``` | Field | Required | Notes | | --------------- | -------- | --------------------------------------------------------------------------------------------- | | `rules[].id` | yes | non-empty, and unique across the **whole document**, not just this node | | `rules[].when` | no | request-scope condition; an entry with no `when` always matches | | `rules[].mode` | no | `ledger`, `shadow` or `route`, overriding the config's top-level `mode:` for matched requests | | `rules[].cache` | no | a response-cache annotation; see below | | `rules[].to` | yes | any node | | `default` | **yes** | the branch taken when nothing matches; labelled `default` on the row | A `when`-less entry that is not last makes every entry after it unreachable, and that is a validation error rather than a silent dead rule.
A rule may enable response caching for the requests it decides. It names an entry in `cache.rules` and may override that entry's mode and TTL within validated bounds. ```yaml - id: chat-turns when: 'trace.step_class = "user_turn"' cache: { rule: chatcache, mode: on, ttl: 10m } to: { tier: cheap } ``` `rule` is required and must name a real `cache.rules` entry. `mode` takes the cache-mode vocabulary `off | shadow | on`, a different set from the rule-level `ledger | shadow | route`. `ttl` is a duration string. The annotation permits the named cache rule to run; it does not read or write the cache itself. A policy without this field behaves as it did before policy-level caching was added. See [caching](/docs/build/caching).
### `select` — filter, then score Use `select` when several tiers could serve the request. TokenTriage removes candidates that do not satisfy `where`, then asks the named selector to choose among the remaining tiers. ```yaml select: from: { tiers: all } where: "candidate.healthy" by: { selector: cheapest_meeting, params: { default_output_tokens: 512 } } on_empty: { tier: frontier } ``` | Field | Required | Notes | | ---------- | -------- | ---------------------------------------------------------------------- | | `from` | yes | exactly one of `{pool: }`, `{tiers: [, ]}`, `{tiers: all}` | | `where` | no | candidate-scope condition, applied per candidate as a hard filter | | `by` | yes | `{selector: , params: {…}}`; there is no default selector | | `on_empty` | no | taken when no candidate survives; absent falls to `on_no_candidate` | Each selector accepts a fixed set of `params`. An unknown parameter fails validation, and the error names the selector. `{tiers: all}` expands in the order tiers are declared. ### `split` — a seeded, sticky draw Use `split` for an A/B test or canary rollout. The same trace stays in the same arm by default; set `sticky: request` when each request should receive its own assignment. ```yaml split: sticky: trace arms: - { id: control, weight: 95, to: { tier: frontier } } - { id: canary, weight: 5, to: { tier: mid } } ``` | Field | Required | Notes | | --------------- | -------- | --------------------------------------------------------------------- | | `sticky` | no | `trace` (the default) or `request` | | `arms` | yes | one to sixteen | | `arms[].id` | yes | unique within this split; two different splits may reuse a name | | `arms[].weight` | yes | a positive integer; zero is refused; weights need not sum to anything | | `arms[].to` | yes | any node, so an arm may hold a whole sub-policy | ### `cascade` — try, grade, escalate Use `cascade` when one request may make multiple provider calls: try a lower-cost tier, grade its response, and use the final target when an earlier response does not meet `accept_when`. ```yaml cascade: stages: - id: try-cheap to: { tier: cheap } judge: true accept_when: "outcome.judge.available and outcome.judge.ok" final: { tier: frontier } ``` | Field | Required | Notes | | ---------------------- | -------- | ----------------------------------------------------------------- | | `stages` | yes | one to four | | `stages[].id` | yes | unique within this cascade | | `stages[].to` | yes | the target for this attempt | | `stages[].accept_when` | yes | outcome-scope condition that decides whether to accept this stage | | `stages[].judge` | no | default `false`; runs the inline judge before the check | | `stages[].timeout_ms` | no | default 60 seconds per stage | | `final` | yes | any node; it is always accepted and has no `accept_when` field | A cascade may not appear beneath another cascade, in a stage or in `final`, or anywhere below either. ### Terminal targets A terminal target ends policy evaluation. Most policies route to a configured tier; the other forms support refusals, passthroughs, weighted pools, and registered extensions. | Target | Shape | Notes | | ----------- | ------------------------------------------------------- | -------------------------------------------------------------------- | | tier | `{ tier: cheap }` | the ordinary target; inherits the tier's generation controls | | model | `{ model: m, upstream: u }` | both keys required; the upstream must be a configured `upstreams:` entry | | pool | `{ pool: cheap_pool }` | shorthand for a `select` over that pool with the `weighted` selector | | refuse | `{ refuse: { status: 429, reason: budget_exhausted } }` | both keys required; any positive status | | passthrough | `{ passthrough: true }` | serve the client's own model unchanged | | custom | `{ target: { kind: , spec: {…} } }` | names a registered non-built-in kind | `passthrough` must be the boolean `true`. A mapping fails validation. A `{ model, upstream }` target routes in `route` mode: the request is sent to the named upstream, under that upstream's own credential, with the model rewritten when it differs from the client's. Because the target names no tier, `decision.tier` on the row is empty — read `decision.upstream` and the model rewrite for its destination. Reporting that groups by tier will not count it. ## Conditions Conditions support only the operators below. They cannot perform arithmetic, call functions, or read values outside the [routing input registry](/docs/build/routing-inputs). | Operator | Applies to | Example | | ----------------- | --------------------------------------------- | ------------------------------------------ | | `<` `<=` `>` `>=` | numbers only | `signal.complexity < 0.35` | | `=` `!=` | any type, both sides the same type | `trace.step_class = "tool_loop"` | | `in ( … )` | any type; the list holds literals only | `trace.agent in ("a", "b")` | | `~` | strings only | `request.model ~ "claude-haiku-*"` | | `exists` | any type; an input or `$param`, not a literal | `dim.tenant exists` | | `and` `or` `not` | conditions | `request.has_tools and not request.stream` | TokenTriage evaluates `not` first, then `and`, then `or`. Use parentheses when you want a different grouping. Within a group, evaluation moves left to right and stops as soon as the result is known (short-circuiting). **Operands** are input references, `$param` references, and literals. Each individual comparison must read at least one input or parameter, so a condition such as `1 < 2` fails validation. A bare boolean input is shorthand for `= true`. **Literals.** Numbers are decimal or exponent form and are all floating point. Strings use double quotes only, with `\n`, `\t`, `\"` and `\\` escapes. Booleans are lowercase `true` and `false`. **Types** are `string`, `number` and `bool`. Both sides of a comparison must have the same type. A `$param` is always a number, so it cannot be compared with a string input. Membership uses **parentheses**, not brackets: `in ("a", "b")`. Square brackets are invalid condition syntax. Glob matching with `~` is **case-sensitive** and follows shell rules: `*` matches a run of characters, `?` matches one, and `*` does not cross a `/`. A malformed glob does not fail validation; it silently matches nothing. Conditions are checked against the place where they appear: - `when` reads request data. - `select.where` reads candidate data. - `cascade.accept_when` reads outcome data. A named predicate is checked each place you use it. For example, a predicate that reads `candidate.healthy` may be valid in `select.where` but is invalid in a rule's `when`. An unknown or out-of-scope input, an undeclared `$param` or `@predicate`, or a predicate cycle fails validation and reports the relevant YAML path. [Routing inputs](/docs/build/routing-inputs) lists the values available in each location. ## Params, predicates, and pools **`params`** give names to numeric values such as a complexity cutoff. Calibration can update a parameter that includes a `calibrate` block. A parameter is always a mapping with a `value`; there is no scalar shorthand. ```yaml params: simple_cut: { value: 0.35, calibrate: { signal: complexity, direction: below } } ``` The `calibrate` block tells `tokentriage calibrate` how to update the parameter. Nothing changes until you run that command and activate its output. A parameter without `calibrate` is valid, but the command will not update it; validation reports a warning to make that visible. Use only `below` or `at_least` for `calibrate.direction`. Other values are not currently rejected. Check the direction in the config diff that `tokentriage calibrate` prints before activating its output. **`predicates`** are reusable conditions referenced as `@name`. TokenTriage checks a predicate against the scope where it is used, as described above. It also rejects cycles among named predicates, including predicates that no node references. **`pools`** are named groups of tiers with relative weights. A member is a tier name and an optional weight. The default weight is `1`, and negative weights fail validation. An empty pool also fails validation. **A weight of `0` does not disable a member; it is treated as the default weight of `1`. Remove the member from the pool when you want to disable it.** ## Limits and what validation rejects | Limit | Value | | -------------- | ----- | | tree depth | 16 | | total nodes | 256 | | cascade stages | 4 | | split arms | 16 | These are common reasons a policy does not validate: | Problem | What to do | | -------------------------------------------------- | ---------------------------------------------------------------------------------------- | | unknown key or selector parameter | check the spelling and the selector's supported parameters | | one node contains two decision forms | move one form into a nested node so each node has exactly one | | a `split` has no document `seed` | add `seed` next to `version` and `root` | | an unconditional rule appears before another rule | move the rule without `when` to the end, or use `default` | | a pool is empty | add at least one tier or remove the pool | | a condition reads an unknown or out-of-scope input | use an input listed for that location in [Routing inputs](/docs/build/routing-inputs) | | a tier or upstream name is unknown | define the name in the config's `tiers:` or `upstreams:` block, or correct the reference | An uncalibrated parameter is the exception: validation reports a warning, but the policy can still load.
Validation reports the first error it finds. It checks `version`, `on_no_candidate`, predicates and their cycles, pools, the tree from `root`, the seed requirement, and finally problems such as unreachable rules. After you fix the first problem, the next validation run may reveal another one later in that order.
TokenTriage also checks tier and upstream names against the config's `tiers:` and `upstreams:` blocks, so a misspelled name fails validation.
TokenTriage reads the `routing.policy` subtree on its own. A YAML anchor defined elsewhere in the file therefore cannot be referenced inside the policy block; validation reports an unknown anchor. Anchors defined and used within the policy block work normally.
## Migrating from `routing.rules` The older first-match `routing.rules` list can be converted into an equivalent policy: 1. Run `tokentriage policy migrate` with the current config. 2. Paste the generated document under `routing.policy`. 3. Delete the old `routing.rules` list, then run `tokentriage validate`. The migration command writes warnings to stderr so its YAML output remains safe to copy: ```console $ tokentriage policy migrate -c configs/tokentriage.example.yaml # tokentriage policy migrate — desugared routing.policy (rev 1c3bf9f) # Source: configs/tokentriage.example.yaml. Paste the document below under `routing:\n policy:` to adopt v2. version: 1 params: rule.simple_to_cheap.threshold: value: 0.35 calibrate: signal: complexity direction: below root: rules: - id: pin-tool-loops when: trace.step_class = "tool_loop" mode: route to: tier: frontier - id: simple-to-cheap when: signal.complexity < $rule.simple_to_cheap.threshold to: tier: cheap - id: agent-scoped when: trace.agent = "summarizer" to: tier: cheap default: tier: frontier ``` Before printing, the command verifies that TokenTriage can load the generated YAML. A legacy rule that compares a signal with a threshold becomes a parameter named `rule..threshold`; non-alphanumeric characters in the rule id become `_`. Existing calibration artifacts can therefore continue to use the same thresholds after migration. Rules that match a step class, agent, model, or header do not have a numeric threshold and do not create parameters. That is why the three rules above produce only one parameter. Migration requires `routing.default_tier`. A config without one is legal in `ledger` mode, but there is no default branch for the generated policy. `policy migrate` stops and asks you to set one. ## What this page does not cover This page describes the policy structure rather than every value it can reference. See [Routing inputs](/docs/build/routing-inputs) for condition data and [Selectors and scorers](/docs/build/routing-selectors) for the primitives a `select` node can name. The surrounding `routing:` settings — `default_tier`, `cooldown`, `cache_affinity`, and `on_internal_error` — are in the [configuration reference](/docs/reference/configuration#routing--classifier).