What routing does not do

Use this page before you enable route mode or build reports from routing data. It covers cases where a request succeeds but the selected route is not applied, settings that are accepted but ignored, and fields that need special handling in reports.

Each limitation explains what happens, what you can observe, and what to do next. If tiers, upstreams, or routing modes are new to you, start with the routing overview. The behaviours below are current as of the updated date shown on this page.

When a recorded route is not applied

The following cases can make the decision data disagree with the request that was served. Check them before treating decision.tier as a record of the request’s actual destination.

{ model, upstream } targets leave decision.tier empty

In route mode, a policy target that names only a model and upstream does send the request to that upstream, under that upstream’s own credential, rewriting the model when it differs from the client’s. What it has no value for is decision.tier, because no tier decided the request: the row carries the rule ID, decision.upstream, and the model rewrite, and decision.tier is empty.

Reporting that groups or filters by decision.tier therefore drops these requests, or files them under an empty tier. Read decision.upstream alongside it. The same applies to a custom target that resolves to a model and upstream; a custom target that resolves to a tier fills decision.tier normally.

What to do instead. If you want these requests to appear in tier-based reports, define a tier for that upstream and model and route to it.

Streaming requests bypass three routing features

Streaming requests bypass the features below. Only cross-dialect translation marks the decision row with decision.fail_safe: true; cascades and shadow duplication leave no feature-specific marker.

Feature	What happens	What to do
cascades	the first stage serves the request as one ordinary call; the row contains no cascade block	use a non-streaming request when cascade evaluation is required
cross-dialect translation	the request falls back to the default upstream that accepts the client’s API format; decision.fail_safe is true	keep streaming traffic on an upstream that uses the same API format
shadow duplication	the request is not sampled and no duplicate row is written	collect comparison data from non-streaming traffic

ledger mode can record a tier that did not serve the request

A request carrying x-tt-tier in ledger mode is served unchanged. The decision row still records the overridden tier in decision.tier, even though that tier did not serve the request. The response contains no decision headers.

What to do instead. Exclude rows where decision.override is true when aggregating ledger-mode traffic by tier. Use decision.upstream when you need the upstream that served the request.

Capabilities routing does not provide

Routing chooses a tier. It is not a retry layer, an active health checker, or the system that accepts and rejects requests against spending and rate limits.

• Retries and hedging. Routing makes the chosen provider call once. It does not retry a failed call or send the same request to several upstreams and use the first response. Your application receives the upstream failure.
• Active health probes. Health is inferred from ordinary request traffic. TokenTriage does not send background probes to test an upstream before routing to it.
• Half-open cooldown recovery. When a cooldown timer expires, the upstream becomes eligible immediately rather than receiving a limited trial request first. Per-minute failure counters are not cleared on expiry, so another failure in the same minute can start a new cooldown.
• Automatic cascade failover. A cascade advances only when its accept_when condition is false. A condition that reads outcome.status can advance after an upstream error, but that is a check in your policy and the failed attempt is still billed.
• Limit enforcement. The ctx.budget.*, ctx.limit.*, and ctx.principal.* values can steer a routing decision. Governance separately decides whether to accept or reject the request in every mode.

Spread traffic and fail over explains the available pool and cooldown behaviour.

Settings and values with narrower behaviour

These settings load successfully, but their runtime behaviour is narrower than their names imply. The last column gives the assumption to use when writing a policy or reviewing its output.

Setting	Current behaviour	How to handle it
classifier.ladder	does not select scorers; it changes only the evidence fingerprint	configure scorers with routing.policy.scorers
classifier.oracle.enabled	does not switch judge calls on or off; api_key_env still feeds both judges and model still feeds shadow duplication, while enabled affects only two validation checks	do not rely on this flag to disable judge calls
least_latency window	is accepted but ignored; the selector always uses the 256 most recent observations	treat the measurement window as fixed
epsilon_greedy reward	is accepted but cannot select another reward source; the selector always reads the graded reward	expect the graded reward to be used
candidate.max_context	has no supplied value, so conditions that read it are false	do not use it in a policy condition
ctx.principal.rpm_remaining_frac	always returns 1, so a condition such as < 0.2 never matches	enforce rate limits through governance rather than steering on this value
calibrate.direction	accepts values other than below or at_least; a typo appears as an unexpected direction in the config diff	use one of the two documented values and check the diff before activation

Reporting fields that need special handling

The request may be served correctly while a cost or revision field supports a narrower conclusion than its name suggests. Apply these rules before using the fields in an aggregate or audit.

cost.usd excludes discarded cascade attempts

The row’s top-level cost.usd covers only the accepted attempt. The cascade block carries a priced entry for every attempt, including the ones that were discarded.

How to calculate the total. Sum the per-stage costs in the cascade block, then add the judge cost. Summing top-level cost.usd across rows under-reports cascade spending.

$0.000000 can mean unpriced rather than free

policy explain prints an unpriced cascade stage as $0.000000. The JSON row and dashboard drawer retain the unpriced flag, and the cascade block records how many stages are unpriced.

How to interpret it. Check the unpriced flag before treating the printed value as zero. An unpriced stage has an unknown cost, not a free one.

policy_rev does not identify the active calibration

policy_rev is a fingerprint of the policy document. Activating a calibration changes the thresholds used by a decision without changing this fingerprint. ctx_ver identifies other runtime state, but it does not identify the calibration overlay either.

How to analyse it. Do not use policy_rev alone to group rows by routing behaviour. Split an analysis at calibration activation times and record the active calibration separately.

Two CLI commands skip corrupt log lines without warning

policy explain and policy preview skip corrupt decision-log lines silently. report, eval, and calibrate print a count of skipped lines to standard error.

What to do. If a row you expect cannot be found, run one of the commands that reports skipped lines before concluding that the row is absent.

Testing and tooling limits

Before enabling attribute routing, ordered fallback, or least_latency, run the policy in shadow mode on your traffic and inspect the resulting decision rows.

How these strategies are covered in automated tests

Eleven named strategies are covered, but not through identical paths. Nine are driven through the proxy and read back through the dashboard API. Attribute routing on trace.* and dim.*, plus ordered fallback with first_eligible, are tested against the policy compiler and evaluator rather than through the complete proxy and dashboard path. least_latency shares the end-to-end pool path exercised through weighted.

This is a difference in automated test coverage, not a statement that the strategies do not route.

Commands and dashboard features unavailable in this version

There is no tokentriage policy test command for running a policy against inline fixture requests. Use tokentriage validate for semantic validation and lint output.

The dashboard’s single-parameter calibration sweep supports legacy routing.rules configurations only. For a v2 routing.policy configuration, use policy preview or the replay API to decide log rows again under a changed policy. The calibrations API returns an explicit error when asked to sweep a v2 policy.

Related product limits

This page covers routing behaviour only. Related limits are documented separately:

• How TokenTriage compares names the gateway-wide capabilities that are not shipped here, including request retry and automatic failover, and says which gateway has each of them today.
• The honesty model explains what conclusions a cost or quality figure supports.
• Air gap and limits lists the content that guardrails cannot inspect.