Route by caller, time, or budget

Two identical requests may need different models because of who sent them, when they arrived, or how much budget remains. You can route on that context without inspecting the prompt.

Before you start, configure at least two tiers and a valid routing policy; Turn routing on covers that setup. After copying an example, validate it and explain one request in shadow mode:

$ tokentriage validate --config tokentriage.yaml
$ tokentriage policy explain --req-id <request-id>

shadow records the choice without changing live traffic. Each example uses an ordered rules list: TokenTriage checks from top to bottom and applies the first match. A tier names a model configuration, such as cheap or frontier. Checking a rule adds no model calls; a refuse action stops the request before any model is called.

Choose the context you have

Start with the row that matches your goal. Each section below stands alone.

Goal	Read this input
give a caller, agent, or requested model its own tier	dim.*, trace.*, or request.*
move scheduled batch work to a cheaper tier	ctx.time.*
change behavior as one named budget runs low	ctx.budget.<id>.*
apply one rule to every caller’s budget	ctx.principal.*

Route by caller, agent, or requested model

Request metadata tells you which customer, workflow step, or model name is involved. Use trace.* for agent activity, dim.* for business dimensions such as tenant or plan, and request.* for fields the client sent.

Business dimensions are off by default. Set ledger.dimensions.enabled: true before routing on dim.*; trace.* and request.* do not need that setting.

routing:
  default_tier: frontier
  policy:
    version: 1
    root:
      rules:
        - id: vip-tenant-frontier
          when: 'dim.tenant = "acme" or dim.tag.plan = "enterprise"'
          to: { tier: frontier }
        - id: batch-agents-cheap
          when: 'trace.agent in ("summarizer", "labeler") and trace.step_class != "tool_loop"'
          to: { tier: cheap }
        - id: haiku-family-pin
          when: 'request.model ~ "claude-haiku-*"'
          to: { tier: cheap }
      default: { tier: frontier }

The VIP rule comes first, so it wins when a request is both enterprise traffic and a batch-agent step. Move the rules only when you intend to change that priority. trace.step_class is user_turn, tool_loop, or provider_state; the second rule keeps tool loops on the default tier.

Explain a recorded request and check that the expected rule matched. The read lines show the values used in the decision:

  • rules[batch-agents-cheap]    matched
      read trace.agent = summarizer
      read trace.step_class = user_turn

Matching syntax and missing values

Write membership with parentheses: in ("a", "b"). Glob matching with ~ is case-sensitive. * matches a run of characters but does not cross /, and ? matches one non-/ character. That means claude-haiku-* matches neither Claude-Haiku-4-5 nor the provider-qualified anthropic/claude-haiku-4-5.

When a client omits a business-dimension header, that dim.* value is missing and its comparison is false. With ledger.dimensions.enabled: false, every dim.* value is missing regardless of which headers the client sends. Routing inputs lists every field and when it may be missing.

Use a quality floor instead when the requirement is “whichever model is good enough” rather than “this caller gets this tier.”

Route by the clock

Batch work that runs overnight can accept a slower, cheaper model that daytime traffic cannot.

routing:
  default_tier: frontier
  policy:
    version: 1
    root:
      rules:
        - id: off-peak-batch-cheap
          when: 'trace.agent = "nightly-batch" and (ctx.time.hour_utc >= 22 or ctx.time.hour_utc < 6)'
          to: { tier: cheap }
      default: { tier: frontier }

The example matches from 22:00 through 05:59 UTC and only for nightly-batch. Keep the agent condition unless you intend to move every request during that window.

Time inputs are always present. ctx.time.hour_utc runs from 0 to 23, ctx.time.weekday uses 0 for Sunday, and ctx.time.unix is the arrival time in epoch seconds. The main trap is timezone conversion: the policy always compares UTC, not local time.

Explain a request near the edge of the window and check the recorded hour. Use a budget rule instead when the goal is to control spend rather than schedule work.

Why a replayed decision gets the same answer

TokenTriage records the request’s arrival time and uses that value during evaluation. Replaying the row under the same policy therefore follows the original time branch, not the branch that matches the current wall clock.

Degrade, then decline, as a budget runs out

When a budget is close to exhausted, the useful behaviour is usually to keep serving on something cheaper, and only stop once there is nothing left.

routing:
  default_tier: frontier
  policy:
    version: 1
    root:
      rules:
        - id: budget-hard-stop
          when: "ctx.budget.default.exhausted"
          to: { refuse: { status: 429, reason: "budget_exhausted" } }
        - id: budget-degrade
          when: "ctx.budget.default.remaining_frac < 0.20"
          to: { tier: cheap }
      default: { tier: frontier }

ctx.budget.<id>.* provides three values for a configured budget: remaining_frac, remaining_usd, and exhausted. The example stops exhausted requests first, then moves requests to cheap after less than 20% remains.

Change the budget id, the 20% threshold, or the refusal response to fit your policy. The id must name a configured budget; an unknown id fails validation.

TokenTriage reads an already-computed budget snapshot, so this rule does not call another service for each request. If budget state is unavailable or suspended, the values are missing, the conditions do not match, and the request uses default instead of being declined.

A declined request returns your status, an X-TT-Refused-By-Policy header, and a body containing the reason. It also writes decision.refused_by_policy to the ledger:

{
  "error": {
    "type": "policy_refused",
    "message": "request refused by routing policy: budget_exhausted",
    "refused_by_policy": "budget_exhausted"
  }
}

This policy declines requests only in route mode. In ledger and shadow, TokenTriage records the plan and still serves the request. Use governance when a spending limit must be enforced in every mode.

Route by the caller’s own budget

The previous section names one budget. This rule reads all budgets that apply to the current caller, so the same condition works for every virtual key.

routing:
  default_tier: frontier
  policy:
    version: 1
    root:
      rules:
        - id: budget-aware
          when: "ctx.principal.budget_remaining_frac < 0.10"
          to: { tier: cheap }
      default: { tier: frontier }

ctx.principal.budget_remaining_frac is the smallest remaining fraction across the caller’s key, user, team, organisation, and global budgets. A key that has spent 95% of its budget reads 0.05 and routes to cheap. This input requires governance.enabled: true.

Anonymous or unresolved traffic has no ctx.principal.* values, so the condition does not match and the request uses default.

Do not route on remaining request-rate capacity yet

ctx.principal.rpm_remaining_frac exists in the policy language, but this build always reports it as fully available. A low-remaining condition on that field does not fire. Use budget_remaining_frac for routing, or use governance to enforce request-rate limits.

Reading ctx.principal.* steers the request in route mode. Use governance when the request must be admitted or declined regardless of the routing mode.

Avoid negating a value that may be missing

Most comparisons are false when a context value is missing, but not can turn a missing value into true. For example, not ctx.budget.default.exhausted matches when budget state is unavailable. Prefer a positive numeric condition such as ctx.budget.default.remaining_frac > 0.10, which is false when the value is missing.

How a missing input behaves has the complete truth table.