# How a routing decision is made

Follow one request from root to an outcome, see how missing inputs behave, and learn what can change the result.

Source: https://tokentriage.com/docs/build/routing-decision

---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
import RoutingWalkDiagram from '../../../components/RoutingWalkDiagram.astro';

Use this page when a request did not follow the routing rule you expected. A routing decision
has two parts: the policy computes an outcome, then the current mode decides whether to apply
that outcome or only record it.

A **policy** is a decision tree. Evaluation starts at `root` and follows **nodes**, or decision
points, until it reaches an outcome. A **tier** is a named routing target that pairs a model
with a configured provider endpoint, called an upstream. The top-level `mode:` setting decides
whether TokenTriage changes the live request.

If tiers and modes are new to you, start with the [Routing overview](/docs/build/routing-overview).
For setup steps, use [Turn routing on](/docs/build/routing). The
[policy language](/docs/build/policy-language) is the full syntax reference.

## Follow one decision from start to finish

This policy excerpt sends requests with a complexity score below `0.35` to the `cheap` tier.
Everything else goes to `frontier`.

```yaml
routing:
  policy:
    version: 1
    params:
      simple_cut: { value: 0.35 }
    root:
      rules:
        - id: simple-to-cheap
          when: 'signal.complexity < $simple_cut'
          to: { tier: cheap }
      default: { tier: frontier }
```

Now take a request whose complexity score is `0.21`. In `ledger` mode, the explanation for
that request looks like this:

```console
$ tokentriage policy explain --req-id tt_simple --log ~/.tokentriage/decisions.jsonl
req_id:     tt_simple
policy_rev: 184bdaf
decision:   rule=simple-to-cheap tier=— upstream=—
path:       root → rules[simple-to-cheap]
trace:
  • rules[simple-to-cheap]       matched
      read signal.complexity = 0.21
      read $simple_cut = 0.35
```

The policy reads `0.21`, compares it with `0.35`, and takes the first matching rule. That rule
names the `cheap` tier. Because the request ran in `ledger` mode, TokenTriage records the rule,
path, and values it read but does not change the request or record a selected tier. That is why
`tier` and `upstream` are dashes. `shadow` would also record the tier it would have used;
`route` would apply that tier to the live request.

## The decision order

When a recorded result differs from what you expected, check these steps in order:

1. **Walk the policy.** Evaluation starts at `root` once per request, before an upstream call.
   Each node either chooses another node or produces an outcome.
2. **Resolve the outcome.** The policy reaches a tier, a refusal, a passthrough, or a fail-safe.
   A cascade is slightly different: it prepares an ordered list of attempts that are evaluated
   as responses come back.
3. **Escalate an unavailable tier.** If the chosen tier is cooling down or cannot accept the
   request's API format, TokenTriage moves toward the more capable end of your tier list. It
   never moves downward.
4. **Apply a client override.** The `x-tt-tier: cheap` header replaces the selected tier and
   labels the rule `override`. An unknown tier name is ignored instead of failing the request.
5. **Keep a conversation with its warm prompt cache.** In `route` mode, cache affinity may keep
   a conversation on the tier whose provider cache is already warm. An explicit client override
   wins over this preference.
6. **Apply the mode.** `route` enforces the outcome. `shadow` and `ledger` leave the live request
   unchanged and record different amounts of decision detail.

The diagram focuses on the policy walk, its four outcomes, and the final mode check. Read it
left to right; the override and cache-affinity steps sit between the chosen tier and the mode
check.

<RoutingWalkDiagram />

## How each node chooses

Each node type answers a different kind of routing question:

- **`rules`** checks entries in order and takes the first one whose `when` condition is true.
  An entry without `when` always matches. Anything after it is unreachable, so validation
  reports an error. If nothing matches, the required `default:` branch is used.
- **`select`** removes candidates that fail its `where` filter, scores the candidates that
  remain, and picks one. [Pick the cheapest candidate that qualifies](/docs/build/routing-cheapest)
  shows this pipeline with an example.
- **`split`** makes a repeatable weighted choice from its seed, sticky key, and node identity.
  The same inputs produce the same assignment.
- **`cascade`** prepares an ordered list of attempts. The request is sent to one stage at a time,
  and the response determines whether the cascade accepts it or tries the next stage.

Conditions evaluate from left to right. With `and` or `or`, evaluation stops as soon as the
result is known. The explanation records only the inputs that were actually read, so it may
show fewer reads than the condition contains.

## How a missing input behaves

An input is **absent** when TokenTriage has no value for it. Examples include a signal that no
scorer emitted, a latency measurement for a new upstream, a missing catalog price, or a
request dimension the client did not send.

Absent is different from zero and `false`; both of those are present values. For strings, an
empty string is treated as absent. Most comparisons involving an absent value are false, but
inequality and negation can produce a surprising true result:

| Condition, where `X` is absent | Result |
|---|---|
| `X exists` | false |
| `X = 0.5` | false |
| `X < 0.5`, `X <= 0.5`, `X > 0.5`, `X >= 0.5` | false |
| `X in ("a", "b")` | false |
| `X ~ "claude-*"` | false |
| `X` on its own, where `X` is a bool | false |
| `X != 0.5` | **true** |
| `not X` | **true** |

The last two rows are the important exceptions. A bare boolean input means `X = true`, so an
absent boolean is false. Negating that result makes `not X` true. A not-equal condition also
evaluates to true when the value is absent.

<Callout type="warning">
`not ctx.limit.daily.exhausted` becomes true when the daily-limit state is unavailable. That
can send traffic to the tier the condition was meant to protect. For an availability or quota
guard, prefer a positive numeric condition such as
`ctx.limit.daily.remaining_frac > 0.1`. If the value is absent, the comparison is false and
the request falls through to your declared fallback.
</Callout>

<Details summary="Tell an absent boolean from a present false">

Use `X exists` to ask whether the input is present. Use `X = false` to match a value that is
present and false; an absent input makes that comparison false. `not X` is true for both an
absent input and a present `false`, so it cannot distinguish them. If the difference matters,
test `exists` first.

</Details>

## The four possible outcomes

The policy walk produces one of four outcomes:

| Outcome | What it means | What `route` does |
|---|---|---|
| tier | send the request to the model and upstream named by the tier | rewrites the model and chooses the upstream |
| typed refusal | decline the request with the status and reason from the policy | returns that status without making an upstream call |
| passthrough | keep the model supplied by the client | sends the request without rewriting the model |
| fail-safe | the decision could not be completed | follows `routing.on_internal_error` and records the cause |

`shadow` does not apply these outcomes to the request, but it records the tier the policy would
have selected. `ledger` also leaves the request unchanged; for a tier outcome, it records the
matched rule and values read but not the would-be tier.

A typed refusal is an intentional policy result and carries `fail_safe: false`. A fail-safe
means TokenTriage could not finish the decision, so it uses the fallback you configured and
records the cause.

<Details summary="When TokenTriage produces a fail-safe">

Evaluation can produce a fail-safe when the policy did not compile, the classifier reported an
internal error, a `select` found no candidate and had no applicable fallback, or every eligible
tier was in cooldown. Enforcement can add one when a decision names an unconfigured tier, a
model rewrite fails, or TokenTriage cannot translate the request to the selected upstream's API
format. [What routing does not do](/docs/build/routing-limits) documents those enforcement gaps.

</Details>

## What can go wrong

The validator cannot catch a spelling mistake in a one-part signal name. For example,
`signal.complexty` loads successfully but is absent on every request. Its condition stays false,
so requests keep taking `default` and routing may appear to have no effect. Dotted names such as
`signal.semantic.legal` are checked against the scorers you configured, so a typo in one of those
names produces a config error.

After editing a policy, send one request that should match and run `tokentriage policy explain`
for its request ID. Check the `path:` line first. If the rule you expected is missing, it did not
fire; the `trace:` and `read` lines show which condition settled the result.

<NextSteps items={[
  { title: 'Explain a routing decision', href: '/docs/build/routing-explain', description: 'Run policy explain and diagnose one request from its path and recorded inputs.' },
  { title: 'Routing inputs', href: '/docs/build/routing-inputs', description: 'Look up every value a condition may read and when that value can be absent.' },
  { title: 'The policy language', href: '/docs/build/policy-language', description: 'Find the full grammar for node kinds, operators, validation, and fallbacks.' },
  { title: 'What routing does not do', href: '/docs/build/routing-limits', description: 'Understand the gaps that can prevent a recorded decision from being applied.' }
]} />