# Route to the cheapest eligible tier Route each request to the lowest-cost tier that meets your health, latency and quality requirements. Source: https://tokentriage.com/docs/build/routing-cheapest --- import Details from '../../../components/Details.astro'; import NextSteps from '../../../components/NextSteps.astro'; import SelectPipelineDiagram from '../../../components/SelectPipelineDiagram.astro'; Use a cost-aware `select` policy when several configured tiers can serve a request and you want the least expensive one that still meets hard requirements. A **tier** is a named model and provider target. TokenTriage handles this in two steps: remove tiers that miss requirements such as health or latency, then estimate the request cost for the rest. The comparison is local and adds no model call or provider spend. In `route` mode, only the chosen tier receives the routed request. This guide assumes [routing is turned on](/docs/build/routing) and your configuration already has at least two tiers. The captured example uses three existing tiers named `cheap`, `mid` and `frontier`, ordered from cheapest to most capable. ## Add the cost-aware policy Replace the `routing:` block in your existing configuration with this policy. Adjust the latency target to match your own service-level requirement. ```yaml routing: default_tier: frontier policy: version: 1 root: select: from: { tiers: all } where: "candidate.latency.p95_ms <= 2000 and candidate.healthy" by: { selector: cheapest_meeting } on_empty: { tier: frontier } ``` Four lines control the decision: - `from` supplies every configured tier as a candidate. - `where` requires a healthy tier whose 95th-percentile latency is no more than two seconds. - `by` tells TokenTriage to choose the lowest estimated request cost among the tiers left. - `on_empty` sends the request to `frontier` if no tier passes the requirements. Save the configuration and validate it before restarting TokenTriage: ```console $ tokentriage validate --config tokentriage.yaml ``` Successful validation ends with `OK: configuration is valid`. It catches unknown fields, invalid expressions and references to tiers that do not exist. ## Understand filter, then score Separating requirements from ranking prevents a cheap but unsuitable tier from winning the comparison. Read the diagram from left to right. `where` decides which candidates are eligible. `by` ranks only the candidates that remain. Rejected candidates appear in `filtered[]`, together with the value that rejected them. Every surviving candidate appears in `scores[]`, whether or not it won. If no candidate passes the filter, the node takes its `on_empty` branch. ## See how request cost is estimated `cheapest_meeting` estimates what the current request would cost at each surviving tier and takes the lowest result: ```text cost = prompt_tokens × input_per_tok + est_output × output_per_tok ``` Prompt tokens come from TokenTriage's request estimate. If no positive estimate is available, it uses `1000`. By default, it assumes the response will contain the same number of tokens as the prompt. A 600-token prompt therefore uses a 600-token response estimate unless you set `default_output_tokens`. Ties go to the tier with the lower rank, meaning the tier that appears earlier in your `tiers:` list. | Parameter | Default | What it does | |---|---|---| | `default_output_tokens` | prompt-token estimate | uses this fixed response-length estimate instead | | `min_quality_lcb` | none | requires a calibrated quality floor; see below | | `quality_metric` | none | names the calibrated metric when a floor is set | **What it costs.** Selection adds no model calls or provider spend. TokenTriage calculates the scores locally from its request estimate and the prices loaded with your configuration. **If a tier has no usable price.** TokenTriage excludes it and marks it `unpriced` in the score table. It does not treat a missing price as zero. A tier you explicitly declared free carries a real `$0` price and can win. If no surviving tier has a usable price, the node takes `on_empty`. ## Verify one decision Restart TokenTriage with the validated configuration and send a representative request. Copy the `x-tt-request-id` response header, then explain that request's routing decision. The request ID in this capture was `tt_select`: ```console $ tokentriage policy explain --req-id tt_select \ --log ~/.tokentriage/decisions.jsonl req_id: tt_select policy_rev: 8067f04 decision: rule=default tier=mid upstream=openai path: root → select[tiers] trace: • select[tiers] selected selector=cheapest_meeting score mid 0.0032 {input_cost=0.0009, output_cost=0.00225} score frontier 0.0189 {input_cost=0.0054, output_cost=0.0135} filtered cheap (candidate.latency.p95_ms=3400) ``` The important lines show why `mid` won: - `cheap` was removed because its 3,400 ms p95 latency exceeded the 2,000 ms limit. - `mid` and `frontier` passed the filter, so both received an estimated cost. - `mid` won because its `$0.0032` score was lower than `frontier`'s `$0.0189`. [Read a routing decision](/docs/build/routing-explain) covers the remaining fields and shows where to find the same explanation in the dashboard. ## Add a measured quality floor Price and latency do not tell you whether a tier produces answers that are good enough for your workload. After you have calibrated a quality metric, replace the `by` block with this one: ```yaml by: selector: cheapest_meeting params: min_quality_lcb: 0.98 quality_metric: output_nonempty ``` **Lower confidence bound (LCB)** is a conservative quality estimate that accounts for measurement uncertainty. A floor of `0.98` admits only tiers whose lower bound is at least `0.98` for `output_nonempty`. The floor controls eligibility, not ranking. TokenTriage removes a tier below the floor before comparing costs, so the floor does not reorder the tiers that pass. A tier with no usable quality measurement is also removed. The explanation distinguishes the two cases: - `quality_below_min_lcb`: the tier was measured and fell below the floor - `quality_absent:`: TokenTriage could not read a usable measurement [Quality-aware routing](/docs/calibration/quality-aware-routing) explains the distinction. [Gate routing on quality](/docs/calibration/gate-routing-on-quality) walks through calibration, configuration and verification from start to finish. ## When nobody qualifies An empty candidate set is an expected outcome. Set `on_empty` on the node so the fallback is visible next to the requirements that can trigger it. TokenTriage handles an empty set in this order: 1. The node's `on_empty` branch, when present. 2. The document-level `on_no_candidate` setting. `default_tier` sends the request to `routing.default_tier`; `refuse` returns a typed `503` with the reason `no_candidate_available`. 3. The default `fail_safe` behavior. TokenTriage sends the request without applying a tier and records that the filter emptied without a configured branch.
A valid selector result includes a score table with the candidate it chose. If a selector returns a choice without that supporting row, TokenTriage treats the result as a fault, sends the request without applying a tier and records the reason. An empty candidate set is different: the decision includes the filter and score results, then follows `on_empty`.
## What this page does not cover Use `weighted_objective` when cost, latency or quality should trade off against one another instead of acting as hard requirements. Its weights, normalization and missing-input behavior are in [the selector reference](/docs/build/routing-selectors#weighted_objective). Spreading traffic across interchangeable upstreams and falling back when one is unhealthy is [pools and fallback](/docs/build/routing-pools). Learning from graded responses is [bandit routing](/docs/build/routing-bandit). Producing the quality measurements used by `min_quality_lcb` starts with [calibration from logs](/docs/calibration/calibrate-from-logs).