Routing selectors and signal sources

Use this reference when you need to configure a select node or understand the values in a routing decision. A tier is a model option you configured. A candidate is a tier still being considered for the current request.

A select node makes its choice in three steps:

1. from builds the candidate list.
2. where removes candidates that do not meet your conditions.
3. by names the selector that chooses one remaining candidate.

This example keeps healthy candidates, then chooses the one with the lowest recently observed latency:

select:
  from: { tiers: all }
  where: "candidate.healthy"
  by: { selector: least_latency, params: { stat: p50 } }
  on_empty: { tier: frontier }

The selector never sees candidates removed by where. If it returns a choice that is missing from its score table, TokenTriage treats the result as a routing fault.

If you are still deciding between rules, selection, splits, and cascades, start with Choose a routing strategy. Use the policy language for the surrounding YAML grammar.

Choose a selector

TokenTriage includes six selectors. Start from the job you need the router to do.

The selectors compare existing candidate data before the provider request is sent. They do not add another model call.

weighted and epsilon_greedy use seeded randomness. The other four selectors are deterministic for the same candidate data.

When selector data is missing

Missing data does not have one universal meaning. Check this behavior before you enable a selector on live traffic.

Selector reference

Each entry shows the smallest valid by block, the data the selector needs, and what to inspect on a recorded decision. Open the scoring details when you need formulas, terms, or tie-breaking.

first_eligible

Use first_eligible when the candidate order expresses your preference. It takes the first candidate that passed where.

by: { selector: first_eligible }

Parameters. None. Adding a parameter prevents the configuration from loading.

Data and missing values. The selector reads only the order of the candidates it receives. It does not exclude candidates or depend on runtime measurements.

Check the decision. Look for selector=first_eligible and a position term beside each candidate. The selected candidate should be the first one that was not filtered.

Use another mechanism when you want to retry after an upstream request fails. first_eligible chooses a tier before the request is sent; it is not a retry policy.

weighted

Use weighted to distribute traffic across a pool in fixed proportions.

by: { selector: weighted }

Parameters. None.

Data and missing values. The selector reads each pool member’s weight. A missing weight or an explicit weight: 0 becomes 1. Candidates built from a tier list have no candidate.pool_weight value.

Check the decision. Look for selector=weighted, then compare each candidate’s weight and probability terms with the pool configuration.

Use first_eligible for a priority order or least_latency when measurements should decide the winner.

least_latency

Use least_latency when recent response time matters more than a fixed priority or traffic share. An upstream is a configured connection to a model provider.

by: { selector: least_latency, params: { stat: p50 } }

Data and missing values. TokenTriage records response-time observations after returning a response to your application. A candidate with no observations stays eligible and carries a cold_start term. If every candidate is cold, the first one wins.

Warm every candidate with representative traffic before relying on this selector. One practical path is to start with weighted, wait until every upstream has observations, and then switch to least_latency.

Check the decision. Look for latency_p50_ms or latency_p95_ms beside every measured candidate and cold_start beside candidates without observations.

Use another selector when price or answer quality matters. least_latency considers neither.

cheapest_meeting

Use cheapest_meeting to choose the least expensive candidate among those that passed where.

by: { selector: cheapest_meeting }

LCB means lower confidence bound: a conservative quality estimate produced by calibration.

Data and missing values. The selector needs input and output prices for each candidate. It excludes an unpriced candidate and records unpriced. A tier you declared free has a genuine $0 estimate, remains eligible, and can win.

Prompt tokens come from signal.token_est. When that value is missing or not positive, the selector estimates 1000 prompt tokens. The output estimate mirrors the prompt estimate unless you set default_output_tokens.

When every candidate is unpriced, the selector chooses nothing and the node follows on_empty. A configured quality floor also excludes candidates whose measured lower bound is missing or below the floor.

Check the decision. Look for input_cost and output_cost on every priced candidate, plus quality_lcb when you configured a floor.

Use weighted_objective when cost is one part of a trade-off rather than the sole score.

score = prompt_tokens × price.input_per_tok + est_output × price.output_per_tok

weighted_objective

Use weighted_objective when no single measurement should decide the route. It combines up to three axes—cost, latency, and quality—with weights you set.

by:
  selector: weighted_objective
  params:
    weights: { cost: 0.5, quality: 0.5 }
    quality_prior: { cheap: 0.6, frontier: 0.9 }

How it chooses. The selector converts each active axis to a value from 0 to 1. It reverses cost and latency so that a better result is always higher, applies your weights, and adds the contributions. The highest total wins.

Data and missing values. The cost axis needs prices and a prompt-token estimate. The latency axis reads candidate.latency.p95_ms. Quality comes from your quality_prior or the calibrated candidate.quality.lcb value.

A candidate missing any axis with a weight above zero is excluded. A weight of zero removes that axis completely, including as a reason to exclude a candidate.

Check the decision. Look for cost, latency, and quality terms. They are the weighted contributions, so the visible terms add up to the final score.

Use cheapest_meeting when price alone should decide. It avoids supplying and maintaining another measurement.

score = Σ weight(axis) × normalized(axis)

epsilon_greedy

Use epsilon_greedy when you have graded outcomes and want the router to keep learning. An arm is one eligible tier. The selector usually chooses the learned tier with the highest pass rate and reserves some requests for exploration.

by:
  selector: epsilon_greedy
  params: { epsilon: 0.05, min_samples: 50, reward: shadow_grade }

How it chooses. An arm is learned when it has a mean and its count reaches min_samples. If at least one arm is learned, a seeded draw against epsilon decides whether to exploit or explore. Exploitation takes the highest learned pass rate. Exploration draws uniformly from all eligible arms.

If no arm is learned, every request explores regardless of epsilon.

Data and missing values. Reward means come from graded shadow-duplication rows for each tier. A tier with no mean, or too few samples, is marked cold and remains available only through exploration. An absent reward_count reads as zero. A configured quality floor can exclude a tier before this choice.

Check the decision. Look for reward_mean and count on each tier. The chosen row carries exploit or explore, plus the reason exploration happened.

Use a fixed selector when you cannot tolerate exploratory traffic or do not yet have a grading pipeline.

Read the decision fields

Run policy explain with a request id to see the selector, every candidate score, and the reason a candidate was filtered or excluded.

$ tokentriage policy explain --req-id tt_route

The selector entries above name the fields that matter for a first check. Open the complete table when you need to interpret a less common term.

Request scorers

Selectors choose a candidate. Request scorers do a different job: they inspect the request before routing and emit signal.* values that rules and filters can read. Configure them under routing.policy.scorers. TokenTriage includes semantic and est_cost; any other name prevents the configuration from loading.

semantic

Use semantic to compare a prompt with example phrases for topics you define. It emits one signal.semantic.<topic> value from 0 to 1 for each topic.

scorers:
  semantic:
    budget_ms: 200
    topics:
      legal:
        exemplars: ["draft a contract clause", "review this NDA"]

Put the threshold in the routing condition, not in the scorer configuration. For example, signal.semantic.legal >= 0.7.

Check the decision. Confirm that the expected signal.semantic.<topic> values are present, then compare them with the threshold in the matched rule.

est_cost

Use est_cost when a routing rule needs an approximate request cost before choosing a tier. It emits signal.est_cost in dollars for the reference tier you name.

scorers:
  est_cost:
    reference_tier: frontier

The signal is absent when TokenTriage cannot price the reference tier. A reference_tier that does not name a configured tier prevents the configuration from loading.

Check the decision. Look for signal.est_cost and confirm that the reference tier’s catalog entry matches the provider and region you expect.

signal.est_cost = (prompt_bytes / chars_per_token) × input_rate
                + est_output_tokens × output_rate

Cascade outcome scorer

An outcome scorer inspects a model response inside a cascade. TokenTriage supplies outcome.refusal_markers, which is available only in a stage’s accept_when condition.

Use = 0 when a stage should accept only responses with no recognized decline phrase:

accept_when: "outcome.status = 200 and outcome.refusal_markers = 0"

The value counts distinct matching patterns, not the number of refusals. A condition such as outcome.refusal_markers >= 2 means that at least two different patterns matched; one strongly worded response can satisfy it.

Context providers

Context providers supply ctx.* values about the current request, time, budget, or governance state. TokenTriage registers them automatically when the required feature is configured.

If budget or governance state is unavailable, the corresponding provider emits no values. A routing condition that depends on an absent value evaluates false. clock reads the request’s arrival time and is always available.

What this page does not cover

This page covers selection mechanisms and the signal producers shipped with routing. Use Routing inputs for every value a policy can read and its absence conditions. Use the policy language for node grammar and terminal targets.

The calibration process that produces candidate.quality.lcb is covered under Calibration and evals.