Routing policy language

Use routing.policy to choose which configured tier handles each request. If this is your first policy, start with the routing overview or follow your first routing rollout. Use this page while writing or troubleshooting the YAML.

Every policy starts at root. Four decision nodes — rules, select, split, and cascade — eventually produce a terminal target such as a tier, refusal, or passthrough. TokenTriage records an explanation each time it evaluates the policy.

Start with a minimal policy

This policy sends requests that use tools to the frontier tier. Every other request goes to cheap.

routing:
  policy:
    version: 1
    root:
      rules:
        - id: tools-to-frontier
          when: "request.has_tools"
          to: { tier: frontier }
      default: { tier: cheap }

After adding a policy, validate the complete config before starting TokenTriage:

$ tokentriage validate --config tokentriage.yaml

The policy document

A policy document supports eight top-level keys. Unknown keys fail validation, so a typo cannot silently turn into a setting that does nothing.

The expanded example below shows how the optional parameter, predicate, and pool blocks connect to the decision tree:

routing:
  policy:
    version: 1
    seed: "ab-1"
    on_no_candidate: default_tier
    params:
      simple_cut:
        { value: 0.35, calibrate: { signal: complexity, direction: below } }
    predicates:
      is_simple: "signal.complexity < $simple_cut"
    pools:
      cheap_pool: { members: [{ tier: cheap }, { tier: cheap-b, weight: 2 }] }
    root:
      rules:
        - id: simple-load-balance
          when: "@is_simple"
          to: { pool: cheap_pool }
      default: { tier: frontier }

routing.policy is mutually exclusive with the older routing.rules list. Setting both fails validation with routing: routing.rules and routing.policy are mutually exclusive; set exactly one (both present).

on_no_candidate decides what happens when a select finds no eligible candidate and has no on_empty branch. The default, fail_safe, forwards the request without choosing a tier and records why. refuse returns a typed 503 with the reason no_candidate_available. default_tier routes to routing.default_tier.

Decision nodes and terminal targets

Choose the decision node that matches what the policy needs to do:

• rules for straightforward if/else routing based on request data.
• select to filter several tiers and choose the best eligible one.
• split for reproducible A/B tests and canary traffic.
• cascade to try an answer, grade it, and escalate when necessary.
• A terminal target to finish with a tier, refusal, passthrough, or another terminal result.

Each node contains exactly one of these forms. For example, do not put both rules and select on the same node. Validation points to the node when two forms are mixed.

rules — first match wins

Use rules for predictable routing based on information already available on the request. The first matching entry wins; default handles every request that did not match.

The list and its default are sibling keys on the node, not nested under a rules: mapping.

rules:
  - id: pin-tool-loops
    when: 'trace.step_class = "tool_loop"'
    mode: route
    to: { tier: frontier }
default: { tier: cheap }

A when-less entry that is not last makes every entry after it unreachable, and that is a validation error rather than a silent dead rule.

- id: chat-turns
  when: 'trace.step_class = "user_turn"'
  cache: { rule: chatcache, mode: on, ttl: 10m }
  to: { tier: cheap }

select — filter, then score

Use select when several tiers could serve the request. TokenTriage removes candidates that do not satisfy where, then asks the named selector to choose among the remaining tiers.

select:
  from: { tiers: all }
  where: "candidate.healthy"
  by: { selector: cheapest_meeting, params: { default_output_tokens: 512 } }
  on_empty: { tier: frontier }

Each selector accepts a fixed set of params. An unknown parameter fails validation, and the error names the selector. {tiers: all} expands in the order tiers are declared.

split — a seeded, sticky draw

Use split for an A/B test or canary rollout. The same trace stays in the same arm by default; set sticky: request when each request should receive its own assignment.

split:
  sticky: trace
  arms:
    - { id: control, weight: 95, to: { tier: frontier } }
    - { id: canary, weight: 5, to: { tier: mid } }

cascade — try, grade, escalate

Use cascade when one request may make multiple provider calls: try a lower-cost tier, grade its response, and use the final target when an earlier response does not meet accept_when.

cascade:
  stages:
    - id: try-cheap
      to: { tier: cheap }
      judge: true
      accept_when: "outcome.judge.available and outcome.judge.ok"
  final: { tier: frontier }

A cascade may not appear beneath another cascade, in a stage or in final, or anywhere below either.

Terminal targets

A terminal target ends policy evaluation. Most policies route to a configured tier; the other forms support refusals, passthroughs, weighted pools, and registered extensions.

passthrough must be the boolean true. A mapping fails validation.

Conditions

Conditions support only the operators below. They cannot perform arithmetic, call functions, or read values outside the routing input registry.

TokenTriage evaluates not first, then and, then or. Use parentheses when you want a different grouping. Within a group, evaluation moves left to right and stops as soon as the result is known (short-circuiting).

Operands are input references, $param references, and literals. Each individual comparison must read at least one input or parameter, so a condition such as 1 < 2 fails validation. A bare boolean input is shorthand for = true.

Literals. Numbers are decimal or exponent form and are all floating point. Strings use double quotes only, with \n, \t, \" and \\ escapes. Booleans are lowercase true and false.

Types are string, number and bool. Both sides of a comparison must have the same type. A $param is always a number, so it cannot be compared with a string input.

Conditions are checked against the place where they appear:

• when reads request data.
• select.where reads candidate data.
• cascade.accept_when reads outcome data.

A named predicate is checked each place you use it. For example, a predicate that reads candidate.healthy may be valid in select.where but is invalid in a rule’s when. An unknown or out-of-scope input, an undeclared $param or @predicate, or a predicate cycle fails validation and reports the relevant YAML path. Routing inputs lists the values available in each location.

Params, predicates, and pools

params give names to numeric values such as a complexity cutoff. Calibration can update a parameter that includes a calibrate block. A parameter is always a mapping with a value; there is no scalar shorthand.

params:
  simple_cut:
    { value: 0.35, calibrate: { signal: complexity, direction: below } }

The calibrate block tells tokentriage calibrate how to update the parameter. Nothing changes until you run that command and activate its output. A parameter without calibrate is valid, but the command will not update it; validation reports a warning to make that visible.

predicates are reusable conditions referenced as @name. TokenTriage checks a predicate against the scope where it is used, as described above. It also rejects cycles among named predicates, including predicates that no node references.

pools are named groups of tiers with relative weights. A member is a tier name and an optional weight. The default weight is 1, and negative weights fail validation. An empty pool also fails validation. A weight of 0 does not disable a member; it is treated as the default weight of 1. Remove the member from the pool when you want to disable it.

Limits and what validation rejects

These are common reasons a policy does not validate:

An uncalibrated parameter is the exception: validation reports a warning, but the policy can still load.

TokenTriage also checks tier and upstream names against the config’s tiers: and upstreams: blocks, so a misspelled name fails validation.

Migrating from routing.rules

The older first-match routing.rules list can be converted into an equivalent policy:

1. Run tokentriage policy migrate with the current config.
2. Paste the generated document under routing.policy.
3. Delete the old routing.rules list, then run tokentriage validate.

The migration command writes warnings to stderr so its YAML output remains safe to copy:

$ tokentriage policy migrate -c configs/tokentriage.example.yaml
# tokentriage policy migrate — desugared routing.policy (rev 1c3bf9f)
# Source: configs/tokentriage.example.yaml. Paste the document below under `routing:\n  policy:` to adopt v2.
version: 1
params:
    rule.simple_to_cheap.threshold:
        value: 0.35
        calibrate:
            signal: complexity
            direction: below
root:
    rules:
        - id: pin-tool-loops
          when: trace.step_class = "tool_loop"
          mode: route
          to:
            tier: frontier
        - id: simple-to-cheap
          when: signal.complexity < $rule.simple_to_cheap.threshold
          to:
            tier: cheap
        - id: agent-scoped
          when: trace.agent = "summarizer"
          to:
            tier: cheap
    default:
        tier: frontier

Before printing, the command verifies that TokenTriage can load the generated YAML. A legacy rule that compares a signal with a threshold becomes a parameter named rule.<rule-id>.threshold; non-alphanumeric characters in the rule id become _. Existing calibration artifacts can therefore continue to use the same thresholds after migration.

Rules that match a step class, agent, model, or header do not have a numeric threshold and do not create parameters. That is why the three rules above produce only one parameter.

Migration requires routing.default_tier. A config without one is legal in ledger mode, but there is no default branch for the generated policy. policy migrate stops and asks you to set one.

What this page does not cover

This page describes the policy structure rather than every value it can reference. See Routing inputs for condition data and Selectors and scorers for the primitives a select node can name. The surrounding routing: settings — default_tier, cooldown, cache_affinity, and on_internal_error — are in the configuration reference.