The routing policy language — TokenTriage Docs

routing.policy is a small, typed, deterministic YAML language. It’s evaluated once per request, before the request is served, producing a plan (a chosen tier or a typed refusal) plus a first-class explanation. A legacy routing.rules config desugars to it and decides byte-identically.

Document structure

routing:
  policy:
    version: 1
    seed: "<salt>"     # required for split; recommended for weighted / epsilon_greedy
    params:            # named scalars; the calibratable surface
      simple_cut: { value: 0.35, calibrate: { signal: complexity, direction: below } }
    predicates:        # named, reusable request conditions, referenced as @name
      is_simple: 'signal.complexity < $simple_cut'
    pools:             # named candidate pools
      cheap_pool: { members: [ { tier: cheap, weight: 1 } ] }
    scorers:           # config-declared signal providers
      semantic: { }
    root: <node>       # the decision tree

The five node kinds

rules — a first-match-wins list of { id, when: '<condition>', to: <target> } with a required default: target.
select — hard-filter then soft-score over candidates: from: { tiers: [...] } | { tiers: all } | { pool: <name> }, an optional where: '<candidate-condition>', by: { selector: <name>, params: {...} }, and an on_empty: target.
split — a seeded, sticky traffic split: sticky: trace | request and arms: [ { id, weight, to: <node> } ].
cascade — stages: [ { id, to, judge?, accept_when: '<outcome-condition>' } ] with a final: target (always accepted — the escalation floor).
Leaf targets — a node may be a bare target: { tier: <name> }, { model: <m>, upstream: <u> }, { pool: <name> }, { refuse: { status, reason } }, { passthrough: {} }, or the generic { target: { kind, spec } }.

The condition language

The operator set is frozen: comparisons < <= > >= = !=; boolean and or not; presence exists; membership in [ ... ]; and glob match on strings with ~ (e.g. request.model ~ "claude-haiku-*"). Operands are input references, $param references, @predicate references, and literals.

Every condition is type-checked against the input registry at its scope — request (when / predicates), candidate (select … where), or outcome (cascade … accept_when). An unknown input, an out-of-scope input, an undeclared $param/@predicate, or a predicate cycle is a path-named config error — never a silently-absent input.

The input registry

Conditions reference three scopes: request inputs (available in when/predicates), candidate inputs (in select … where), and outcome inputs (in cascade … accept_when). The most-used request inputs are signal.<key> (classifier signals), request.* (the incoming request shape), trace.step_class/trace.agent (agentic context), and ctx.budget.*/ctx.limit.* (live governance state for budget-aware routing). The full closed list is below.

The full input registry (every input, all three scopes)

Request scope: request.model, .dialect, .endpoint, .stream, .prompt_bytes, .messages, .system_bytes, .has_tools, .tool_count, .tool_result_count, .max_tokens, .reasoning_effort; trace.agent, .step_class, .step_class_source, .trace_id/.step_id/.parent_id/.w3c; dim.user/.tenant/.feature, dim.tag.<key>; signal.<key>; ctx.time.hour_utc/.weekday/.unix; ctx.budget.<id>.remaining_frac/.remaining_usd/.exhausted; ctx.limit.<id>.remaining_frac/.exhausted/.suspended; ctx.principal.budget_remaining_frac/.rpm_remaining_frac/.exhausted.
Candidate scope: candidate.tier/.upstream/.model/.provider, .rank, .max_context, .pool_weight, .price.input_per_tok/.output_per_tok (absent when unpriced), .latency.p50_ms/.p95_ms (absent on a cold upstream), .reward_mean/.reward_count (absent on a cold tier), .healthy.
Outcome scope: outcome.status, .output_tokens, .response_bytes, .latency_ms, .finish_reason, .judge.ok/.judge.available, .refusal_markers.

Worked examples

Complexity routing — the calibratable default:

params: { simple_cut: { value: 0.35, calibrate: { signal: complexity, direction: below } } }
root:
  rules:
    - { id: to-cheap, when: 'signal.complexity < $simple_cut', to: { tier: cheap } }
  default: { tier: frontier }

Cascade with an inline judge — try cheap, grade, escalate:

root:
  rules:
    - id: tools-cascade
      when: 'request.has_tools'
      to:
        cascade:
          stages:
            - id: try-cheap
              to: { tier: cheap }
              judge: true
              accept_when: 'outcome.refusal_markers = 0 and outcome.judge.available and outcome.judge.ok'
          final: { tier: frontier }
  default: { tier: mid }

Budget degrade → refuse — order the hard refusal before the soft degrade:

root:
  rules:
    - { id: budget-refuse,  when: 'ctx.budget.default.exhausted', to: { refuse: { status: 429, reason: budget_exhausted } } }
    - { id: budget-degrade, when: 'ctx.budget.default.remaining_frac < 0.20', to: { tier: cheap } }
  default: { tier: frontier }

A/B / canary split:

seed: "ab-1"
root:
  split:
    sticky: trace
    arms:
      - { id: control, weight: 95, to: { tier: frontier } }
      - { id: canary,  weight: 5,  to: { tier: mid } }

Reading a decision’s explanation

Every routed row carries decision.policy_rev (a 7-hex policy revision) and decision.explain, built on the hot path for every request — not optional telemetry. The explanation records the node path walked, a trace[] of visited nodes (with the exact inputs each condition read, in first-read order; a select’s per-candidate scores[] and per-candidate filtered[] exclusion reasons; and a deterministic ctx_ver), any random rolls[] (self-recomputable), a cascade block on cascade rows, and fail_safe/fail_safe_why (non-empty only when a disclosed per-request fail-safe engaged).

Read it back from the dashboard record drawer, over GET /tt/api/v1/requests/{id}, or:

$ tokentriage policy explain --req-id <id> --log ~/.tokentriage/decisions.jsonl

Migrate from legacy rules

A legacy routing.rules list is a special case of the language. policy migrate prints the desugared document to stdout (warnings go to stderr, keeping stdout paste-clean):

$ tokentriage policy migrate -c tokentriage.yaml > policy.snippet.yaml

Then paste it under routing.policy: and delete routing.rules: (the two are mutually exclusive). The command round-trips (it refuses to print anything that doesn’t decode back to an identical document) and the migration decides byte-identically to the legacy config — existing calibration artifacts keep applying, because each legacy rule id resolves to an auto-param. One limitation: a ledger-mode config that omits routing.default_tier has no valid standalone v2 form, so policy migrate refuses it with an actionable message. Full detail in docs/POLICY_MIGRATION.md.