How a routing decision is made

Use this page when a request did not follow the routing rule you expected. A routing decision has two parts: the policy computes an outcome, then the current mode decides whether to apply that outcome or only record it.

A policy is a decision tree. Evaluation starts at root and follows nodes, or decision points, until it reaches an outcome. A tier is a named routing target that pairs a model with a configured provider endpoint, called an upstream. The top-level mode: setting decides whether TokenTriage changes the live request.

If tiers and modes are new to you, start with the Routing overview. For setup steps, use Turn routing on. The policy language is the full syntax reference.

Follow one decision from start to finish

This policy excerpt sends requests with a complexity score below 0.35 to the cheap tier. Everything else goes to frontier.

routing:
  policy:
    version: 1
    params:
      simple_cut: { value: 0.35 }
    root:
      rules:
        - id: simple-to-cheap
          when: 'signal.complexity < $simple_cut'
          to: { tier: cheap }
      default: { tier: frontier }

Now take a request whose complexity score is 0.21. In ledger mode, the explanation for that request looks like this:

$ tokentriage policy explain --req-id tt_simple --log ~/.tokentriage/decisions.jsonl
req_id:     tt_simple
policy_rev: 184bdaf
decision:   rule=simple-to-cheap tier=— upstream=—
path:       root → rules[simple-to-cheap]
trace:
  • rules[simple-to-cheap]       matched
      read signal.complexity = 0.21
      read $simple_cut = 0.35

The policy reads 0.21, compares it with 0.35, and takes the first matching rule. That rule names the cheap tier. Because the request ran in ledger mode, TokenTriage records the rule, path, and values it read but does not change the request or record a selected tier. That is why tier and upstream are dashes. shadow would also record the tier it would have used; route would apply that tier to the live request.

The decision order

When a recorded result differs from what you expected, check these steps in order:

1. Walk the policy. Evaluation starts at root once per request, before an upstream call. Each node either chooses another node or produces an outcome.
2. Resolve the outcome. The policy reaches a tier, a refusal, a passthrough, or a fail-safe. A cascade is slightly different: it prepares an ordered list of attempts that are evaluated as responses come back.
3. Escalate an unavailable tier. If the chosen tier is cooling down or cannot accept the request’s API format, TokenTriage moves toward the more capable end of your tier list. It never moves downward.
4. Apply a client override. The x-tt-tier: cheap header replaces the selected tier and labels the rule override. An unknown tier name is ignored instead of failing the request.
5. Keep a conversation with its warm prompt cache. In route mode, cache affinity may keep a conversation on the tier whose provider cache is already warm. An explicit client override wins over this preference.
6. Apply the mode. route enforces the outcome. shadow and ledger leave the live request unchanged and record different amounts of decision detail.

The diagram focuses on the policy walk, its four outcomes, and the final mode check. Read it left to right; the override and cache-affinity steps sit between the chosen tier and the mode check.

How each node chooses

Each node type answers a different kind of routing question:

• rules checks entries in order and takes the first one whose when condition is true. An entry without when always matches. Anything after it is unreachable, so validation reports an error. If nothing matches, the required default: branch is used.
• select removes candidates that fail its where filter, scores the candidates that remain, and picks one. Pick the cheapest candidate that qualifies shows this pipeline with an example.
• split makes a repeatable weighted choice from its seed, sticky key, and node identity. The same inputs produce the same assignment.
• cascade prepares an ordered list of attempts. The request is sent to one stage at a time, and the response determines whether the cascade accepts it or tries the next stage.

Conditions evaluate from left to right. With and or or, evaluation stops as soon as the result is known. The explanation records only the inputs that were actually read, so it may show fewer reads than the condition contains.

How a missing input behaves

An input is absent when TokenTriage has no value for it. Examples include a signal that no scorer emitted, a latency measurement for a new upstream, a missing catalog price, or a request dimension the client did not send.

Absent is different from zero and false; both of those are present values. For strings, an empty string is treated as absent. Most comparisons involving an absent value are false, but inequality and negation can produce a surprising true result:

Condition, where X is absent	Result
X exists	false
X = 0.5	false
X < 0.5, X <= 0.5, X > 0.5, X >= 0.5	false
X in ("a", "b")	false
X ~ "claude-*"	false
X on its own, where X is a bool	false
X != 0.5	true
not X	true

The last two rows are the important exceptions. A bare boolean input means X = true, so an absent boolean is false. Negating that result makes not X true. A not-equal condition also evaluates to true when the value is absent.

Tell an absent boolean from a present false

Use X exists to ask whether the input is present. Use X = false to match a value that is present and false; an absent input makes that comparison false. not X is true for both an absent input and a present false, so it cannot distinguish them. If the difference matters, test exists first.

The four possible outcomes

The policy walk produces one of four outcomes:

Outcome	What it means	What route does
tier	send the request to the model and upstream named by the tier	rewrites the model and chooses the upstream
typed refusal	decline the request with the status and reason from the policy	returns that status without making an upstream call
passthrough	keep the model supplied by the client	sends the request without rewriting the model
fail-safe	the decision could not be completed	follows routing.on_internal_error and records the cause

shadow does not apply these outcomes to the request, but it records the tier the policy would have selected. ledger also leaves the request unchanged; for a tier outcome, it records the matched rule and values read but not the would-be tier.

A typed refusal is an intentional policy result and carries fail_safe: false. A fail-safe means TokenTriage could not finish the decision, so it uses the fallback you configured and records the cause.

When TokenTriage produces a fail-safe

Evaluation can produce a fail-safe when the policy did not compile, the classifier reported an internal error, a select found no candidate and had no applicable fallback, or every eligible tier was in cooldown. Enforcement can add one when a decision names an unconfigured tier, a model rewrite fails, or TokenTriage cannot translate the request to the selected upstream’s API format. What routing does not do documents those enforcement gaps.

What can go wrong

The validator cannot catch a spelling mistake in a one-part signal name. For example, signal.complexty loads successfully but is absent on every request. Its condition stays false, so requests keep taking default and routing may appear to have no effect. Dotted names such as signal.semantic.legal are checked against the scorers you configured, so a typo in one of those names produces a config error.

After editing a policy, send one request that should match and run tokentriage policy explain for its request ID. Check the path: line first. If the rule you expected is missing, it did not fire; the trace: and read lines show which condition settled the result.