# Route by prompt complexity

Send simple prompts to a cheap tier and hard ones to a strong tier with one policy condition, rolled out shadow-first.

Source: https://tokentriage.com/docs/recipes/route-by-prompt-complexity

---
import Callout from '../../../components/Callout.astro';
import NextSteps from '../../../components/NextSteps.astro';

Route cheap-looking prompts to a cheap tier and hard ones to a strong tier by branching a
routing policy on the built-in `signal.complexity` score — and roll it out **shadow-first** so
a report proves the savings before a single served response changes.

## Prerequisites

- TokenTriage running with **at least two tiers** — a cheap one and a strong one — defined in
  `tiers:` (see [Providers](/docs/build/providers)).
- The **built-in classifier ladder** enabled (the default). It emits `signal.complexity`, a
  `[0,1]` score, per request.
- Familiarity with the [policy language](/docs/build/policy-language) — this recipe assumes you
  know what a policy node is.

## Steps

### 1. Confirm your tiers, cheapest → most capable

`tiers:` is an ordered list; the names are free-form and are what a policy target references. A
minimal two-tier setup:

```yaml
tiers:
  - name: cheap
    upstream: anthropic
    model: claude-haiku-4-5
  - name: frontier
    upstream: anthropic
    model: claude-opus-4-8
```

### 2. Add a `routing.policy` that branches on complexity

The policy tree has **five node kinds**: `rules` (first-match-wins), `select` (hard-filter →
soft-score over candidates), `split` (seeded A/B/canary), `cascade` (try-grade-escalate), and
bare **leaf targets** (`{ tier }`, `{ pool }`, `{ refuse }`, `{ passthrough }`, `{ model,
upstream }`). Complexity routing is the simplest case — a single `rules` node whose one rule
sends anything below a threshold to `cheap` and lets everything else fall through to the
default:

```yaml
routing:
  policy:
    version: 1
    params:
      simple_cut:
        value: 0.35
        calibrate: { signal: complexity, direction: below }
    root:
      rules:
        - id: simple-to-cheap
          when: 'signal.complexity < $simple_cut'
          to: { tier: cheap }
      default: { tier: frontier }
```

What each piece is doing, all verified against the frozen grammar:

- `signal.complexity` is a genuine `signal.<key>` input emitted by the ladder; `$simple_cut` is a
  `$param` reference; `<` is one of the six comparison operators (`<`, `<=`, `>`, `>=`, `=`,
  `!=`).
- `default:` is **required** on every `rules` node — it is the target when no rule matches.
- The `calibrate` block makes `$simple_cut` a fittable param (see step 5); it is inert until you
  run the fitter.

<Callout type="note">
`routing.policy` and the legacy `routing.rules` list are **mutually exclusive** — a config sets
exactly one. If your config still uses `routing.rules`, run `tokentriage policy migrate -c
config.yaml` to print the byte-identical desugared policy, then paste it under
`routing.policy`.
</Callout>

### 3. Roll out in `shadow`, not `route`

Set the top-level trust-ladder mode to `shadow`. In `shadow`, TokenTriage records the
*would-route* decision (and can duplicate to the counterfactual tier and grade it) **without
touching any served response** — the served model is still whatever the client asked for.

```yaml
mode: shadow          # ledger | shadow | route  — the trust ladder
```

Send some real traffic, then read a decision back with the explain command:

```console
$ tokentriage policy explain --req-id 01J8Z... --log ./decisions.log
```

`policy explain` prints **human-readable text** (not JSON) — the path taken and the exact
inputs each node read, in first-read order. A simple prompt (illustrative):

```console
req_id:     01J8Z...
policy_rev: 3f9a1c7
decision:   rule=simple-to-cheap tier=cheap upstream=openai-cheap
path:       root → rules[simple-to-cheap]
trace:
  • rules[simple-to-cheap]       matched
      read signal.complexity = 0.21
      read $simple_cut = 0.35
```

A hard prompt instead falls through to `default` (illustrative):

```console
req_id:     01J9A...
policy_rev: 3f9a1c7
decision:   rule=default tier=frontier upstream=openai-frontier
path:       root → rules[default]
trace:
  • rules[simple-to-cheap]       skipped (when=false)
      read signal.complexity = 0.71
      read $simple_cut = 0.35
  • rules[default]               matched
```

### 4. Prove the savings before enforcing

Because you are in `shadow`, the decision log holds real *would-route* decisions you can score
offline. Re-decide the log under the same (or a draft) policy to see how many requests would move
and the cost delta:

```console
$ tokentriage policy preview --config ./config.yaml --log ./decisions.log
```

`policy preview` re-decides each recorded row through the real evaluator and reports changed /
cost-delta / by-rule. It always emits two approximation labels (a decision is re-simulated, not
re-served) and discloses per-row any draft-referenced input a row did not record — so you never
get a confident wrong delta.

### 5. (Optional) Calibrate the cut to a target frontier share

Instead of guessing `0.35`, fit `$simple_cut` from the log to hit a target share of frontier
traffic. A fit **writes a dormant candidate artifact** (under `~/.tokentriage/calibrations`);
it does **not** edit your YAML, and it stays **inert until you activate it**:

```console
$ tokentriage calibrate -c ./config.yaml --log ./decisions.log --target-frontier-pct 0.30
fitted complexity threshold: 0.28400
config-diff (effect when active):
  param simple_cut               complexity below: 0.35000 -> 0.28400
wrote candidate artifact: cal_20260809T183520Z_d5db85f2 (status=candidate)
activate it with: tokentriage calibrate --activate cal_20260809T183520Z_d5db85f2
```

Nothing affects routing until you promote the candidate by id:

```console
$ tokentriage calibrate --activate cal_20260809T183520Z_d5db85f2 --config ./config.yaml
```

### 6. Promote to `route`

Once the shadow report shows the split you want, flip the mode to enforce it. Now the matched
rule actually rewrites the model / picks the tier:

```yaml
mode: route
```

## Why this is safe

- **Shadow-first is the point.** The `ledger → shadow → route` ladder means you observe and price
  every would-route decision, then prove the savings on real traffic, and only then enforce — no
  served response changes until the last step.
- **A missing signal fails safe.** This is **absent-input comparison semantics**: if
  `signal.complexity` is ever absent, the ordered comparison `signal.complexity < $simple_cut`
  evaluates to **false** — the rule **does not fire** and the request falls through to the
  `default` (`frontier`), your strong tier. Complexity routing therefore never downgrades a
  request on a missing signal.
- **Every decision is explainable.** The `read` block records exactly which inputs the condition
  saw, in first-read order, on the decision-log row — so a "why did this go cheap?" question is
  answered from the record, not reconstructed.

<Callout type="warning">
A `signal.<key>` name is not validation-checked against a live scorer — a typo like
`signal.complexty` is not a config error; by absent-input semantics the rule just never fires
and every request falls to `default`. Confirm your rule actually matches in `shadow` (via
`policy explain`) before promoting to `route`.
</Callout>

<NextSteps items={[
  { title: 'Route on what the prompt looks like', href: '/docs/build/routing-by-prompt', description: 'The mechanism behind this recipe, plus topic similarity and expected-cost routing.' },
  { title: 'The policy language', href: '/docs/build/policy-language', description: 'The full grammar — five node kinds, the frozen condition operators, and what validation rejects.' },
  { title: 'Cost optimization', href: '/docs/build/cost-optimization', description: 'Combine complexity routing with caching and cascades to cut spend while keeping quality.' },
  { title: 'The honesty model', href: '/docs/concepts/honesty-model', description: 'Why an absent input excludes rather than fabricates, and why the explanation is built on every decision.' },
  { title: 'Gate routing on a proven quality floor', href: '/docs/calibration/gate-routing-on-quality', description: 'Go beyond a complexity guess: require a calibrated quality lower bound before the cheap tier is even eligible.' },
]} />