# Choose a routing strategy

Choose a routing strategy from your goal, the information available at decision time, and the cost of making that decision.

Source: https://tokentriage.com/docs/build/routing-strategies

---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';

If you are unsure where to start, use a rule and test it in shadow mode. A rule makes one
model call and is easy to explain when something about the request or caller tells you where
it should go. Shadow mode records the decision without changing live traffic.

Use selection when you need to balance several measurements, a split when you are running an
experiment, and a cascade when you must inspect an answer before deciding whether to try a
stronger model.

A **tier** is a model option you have configured, such as a lower-cost default and a stronger
fallback. A **routing policy** is the set of decisions that chooses among those tiers. [How a
routing decision is made](/docs/build/routing-decision) shows how TokenTriage walks that policy.

## Four ways to make the decision

TokenTriage policies use four node kinds. Each one answers the routing question from different
information.

| Node kind | Use it when | What it decides from |
|---|---|---|
| `rules` | one condition can express the choice | the prompt, caller, time, budget or other request context |
| `select` | several eligible models need to be compared | price, latency, measured quality or another score |
| `split` | you want a stable traffic allocation | a seeded assignment that keeps related requests together |
| `cascade` | you need to inspect an answer before accepting it | the answer from one model and the result of an acceptance check |

## Start from the job

Find the row that sounds like your problem. Start with the plain-language strategy in the
middle column; the configuration names appear in backticks.

| Your goal | Start with | Build it |
|---|---|---|
| route from the prompt's complexity, topic or estimated cost | a prompt rule: `rules` using `signal.*` | [Route on the prompt](/docs/build/routing-by-prompt) |
| route by team, agent, feature, time or remaining budget | a context rule: `rules` using `trace.*`, `dim.*` or `ctx.*` | [Route on context](/docs/build/routing-by-context) |
| choose the cheapest qualified model, or balance cost, latency and quality | scored selection: `select` using `cheapest_meeting` or `weighted_objective` | [Pick the cheapest candidate that qualifies](/docs/build/routing-cheapest) |
| spread traffic, prefer the fastest provider or follow an ordered fallback | pool selection: `select` using `weighted`, `least_latency` or `first_eligible` | [Spread traffic and fail over](/docs/build/routing-pools) |
| send a stable share of traffic to a new policy | a traffic split: `split` | [Split traffic for an experiment](/docs/build/routing-splits) |
| try a lower-cost model, check its answer and escalate when needed | an answer cascade: `cascade` | [Escalate when the answer falls short](/docs/build/routing-cascades) |
| learn which model to prefer from graded outcomes | learned selection: `select` using `epsilon_greedy` | [Let the router learn from outcomes](/docs/build/routing-bandit) |

You can combine these strategies. For example, a policy can route by team, send 5% of one
team's traffic to a new branch, and use a cascade inside that branch. [The policy
language](/docs/build/policy-language) shows how those nodes fit together.

## Compare cost and operational requirements

Compare model attempts first. Rules, selection and splits send each request to one target
model. A cascade can make several attempts and may also pay for a judge.

| Node kind | Target-model attempts | Other work | Data it needs |
|---|---|---|---|
| `rules` | one | evaluate a condition; prompt rules may also classify the request | request and context fields named by the condition |
| `select` | one | filter and score the eligible tiers | candidate tiers and the price, latency, quality or reward data used by the selector |
| `split` | one | assign the request to a stable traffic group | the split seed and the identity used to keep related requests together |
| `cascade` | one or more | check each answer; a model-based judge adds another call | the model answer and an acceptance check |

For an escalated request, a two-tier cascade pays for the first model attempt, the judge when
the judge is another model, and the stronger model attempt. It saves money only when enough
requests stop before that stronger attempt. [Escalate when the answer falls
short](/docs/build/routing-cascades) calculates the break-even acceptance rate.

<Callout type="key">

Start with `rules` when request or caller data can express the choice. Rules are usually the
least expensive option and the easiest to explain and debug. Move to `select` when one
condition cannot express the trade-off, use `split` for an experiment, and use `cascade` only
when you need to inspect an answer before deciding whether to escalate.

</Callout>

## How to tell it is working

A routing decision and a useful outcome are different things. Each request records an
explanation in the decision ledger. Inspect that explanation first, then use reports and
evaluations to decide whether the strategy improved cost, latency or answer quality.

| Node kind | Read this | It is working when |
|---|---|---|
| `rules` | `path` and `read` in `tokentriage policy explain` | the expected rule appears and the recorded input has the value you expected |
| `select` | `score` and `filtered` lines | the intended candidate ranks first and every excluded candidate has a stated reason |
| `split` | the `roll` line and `tokentriage report` grouped by rule | traffic approaches the configured percentages and one conversation stays in the same group |
| `cascade` | the `cascade:` block | the acceptance and escalation rates match the break-even assumption you used |

Across all four, `tokentriage report` shows the observed cost change. `tokentriage eval` shows
whether answer quality held up when you have configured the outcomes or evaluators it needs.
If a strategy routes requests as expected but does not change cost, revisit the assumption
behind it and check that the relevant requests have price data.

## Use these after simpler strategies

Two strategies need more evidence or more calls than a rule.

**Learned routing** needs graded outcomes: a recorded score that says whether an answer was
good enough. While it gathers evidence, it intentionally tries alternatives on some requests.
Expect early routing to be less predictable than a fixed rule.

**Cascades** can increase your bill because one request may produce several model attempts.
Use a cascade when the request alone does not predict answer quality well enough. Prefer a
rule when information available before the first model call can make the choice.

<Details summary="How the routing strategies are covered by the product's test suite">

Most strategies are exercised by sending requests through the proxy and reading the resulting
explanation from the dashboard API. This covers complexity, semantic topic, cost-aware
selection, expected-cost routing, cascades, pool load balancing, traffic splits, learned
routing, budget handling and time-of-day routing.

Attribute routing on `trace.*` and `dim.*`, plus ordered fallback with `first_eligible`, are
tested against the evaluator and policy compiler rather than in dedicated proxy-to-dashboard
scenarios. `least_latency` shares the end-to-end pool path exercised by `weighted`, but it does
not have a dedicated proxy-to-dashboard scenario of its own.

Test ordered fallback and `least_latency` in shadow mode on your traffic before enabling them.

</Details>

## What this page does not cover

This page helps you choose a strategy; it does not estimate savings for your workload.
`tokentriage report` measures the cost change in your recorded traffic, while `tokentriage
eval` checks answer quality when its required evidence is available. [Turn routing
on](/docs/build/routing) explains how to collect those results safely. Use [calibration and
evals](/docs/calibration/overview) to decide whether a lower-cost tier is good enough for the
requests it receives.

<NextSteps items={[
  { title: 'Route on what the prompt looks like', href: '/docs/build/routing-by-prompt', description: 'Build the simplest starting point from prompt complexity, topic or estimated cost.' },
  { title: 'Pick the cheapest candidate that qualifies', href: '/docs/build/routing-cheapest', description: 'Compare candidates when one rule cannot express the cost, latency and quality trade-off.' },
  { title: 'Escalate when the answer falls short', href: '/docs/build/routing-cascades', description: 'Calculate judge and escalation costs before enabling a cascade.' },
  { title: 'How a routing decision is made', href: '/docs/build/routing-decision', description: 'Learn the evaluation order and what happens when a policy input is missing.' }
]} />