GitHubBook a demoStart routing
Browse docs

Example config

One complete, production-shaped tokentriage.yaml you can copy, trim, and validate — every block active and every line commented, from upstreams and tiers through routing, cache, and governance.

Last updated

On this page

The Configuration reference documents every field one at a time. This page is the other half: a single, coherent config with every block active and every line commented, shaped the way a real deployment looks. Copy it, delete the blocks you don’t need, and run tokentriage validate --config tokentriage.yaml — it fails loudly on an unknown key or a bad combination, so a trimmed copy stays honest.

The complete config

# tokentriage.yaml — a production-shaped starting point.
# Boots in shadow mode: every decision is computed and logged, nothing is enforced.

version: 1                      # config schema version — required, currently 1

mode: shadow                    # ledger | shadow | route — the trust ladder.
                                #   ledger: observe only, zero request mutation
                                #   shadow: + compute & log would-route/would-cache/would-refuse
                                #   route:  + enforce (promote here once `eval` proves out)

server:
  listen: "0.0.0.0:8787"        # data plane — proxies your LLM traffic. Serves plaintext
                                # HTTP; terminate TLS in front of it in production.
  admin:  "127.0.0.1:9090"      # admin plane — dashboard, /metrics, /health, /tt/api/*.
                                # Loopback-bound so dashboard auth may be "off" (below).
  max_body_bytes: 10485760      # bounded request buffer; default 10 MiB

upstreams:                      # named upstreams — a map, not a fixed pair.
  anthropic:
    base_url: https://api.anthropic.com
    dialect: anthropic          # openai | anthropic — the wire dialect to speak
    api_key_env: ANTHROPIC_API_KEY   # env-var NAME (never the secret); when set it
                                     # replaces inbound credentials on the way upstream
  openai:
    base_url: https://api.openai.com/v1
    dialect: openai
    api_key_env: OPENAI_API_KEY

tiers:                          # ordered cheapest -> most capable; names are free-form.
  - name: cheap                 # the tier `routing` downgrades simple traffic to
    upstream: anthropic         # which upstream serves this tier
    model: claude-haiku-4-5     # the model this tier rewrites requests to
  - name: frontier              # the capable default
    upstream: anthropic
    model: claude-opus-4-8

routing:
  default_tier: frontier        # matches-nothing target; also the fail-safe on classifier
                                # error (routing never downgrades on error)
  on_internal_error: passthrough  # passthrough (default) | default_tier
  rules:                        # ordered, first-match-wins, evaluated per request.
                                # (`rules:` and `policy:` are mutually exclusive — pick one.)
    - id: pin-tool-loops        # stable id; appears verbatim in the decision ledger
      match:
        step_class: tool_loop   # user_turn | tool_loop | provider_state
      route: frontier
    - id: simple-to-cheap
      match:
        signal: complexity      # a named classifier signal
        below: 0.35             # threshold; `tokentriage calibrate` can rewrite this
      route: cheap
  cooldown:                     # temporarily divert traffic off a failing upstream
    enabled: true
    seconds: 30                 # cool-off duration
    failure_threshold: 0.5      # trip when >50% of requests fail ...
    min_requests: 5             # ... over at least 5 requests in the current minute

ledger:
  decision_log:
    path: ~/.tokentriage/decisions.jsonl   # JSONL is the system of record; everything
                                           # else (index, dashboard) rebuilds from it
  redact:
    prompt: hash                # hash (default) | none — "none" stores raw prompt text
  dimensions:
    enabled: true               # capture x-tt-user / x-tt-tenant / x-tt-feature / x-tt-tag-*
                                # attribution. Required for real cache/governance isolation.

dashboard:
  enabled: true                 # mount the dashboard + /tt/api/v1 on the admin plane
  auth:
    mode: off                   # off | token | users. "off" is allowed ONLY on a loopback
                                # admin bind (as above); a non-loopback bind rejects it.
  index:
    enabled: true               # the SQLite read index; DERIVED, delete + rebuild any time

cache:                          # TokenTriage's OWN response cache. Absent ⇒ off.
  enabled: true                 # master gate
  scope: [tenant]               # closed vocab: tenant|user|feature|trace|upstream-key.
                                # Needs ledger.dimensions.enabled (set above) to isolate.
  rules:                        # ordered, first-match — the routing-rules idiom
    - id: chat-exact
      match:
        endpoint: chat          # chat | responses (the cacheable endpoints in v1)
      mode: shadow              # off | shadow | on. shadow records would-hits and forwards
                                # anyway — flip to "on" once the hit-rate proves out.
      strategy: exact           # exact | normalized | semantic (or a chain)
      ttl: 1h

governance:                     # virtual keys & per-tenant limits. Absent ⇒ off.
  enabled: true                 # master gate
  require_key: optional         # off | optional | required. RESTART-required to change.
                                # required = default-deny (anonymous traffic gets an honest 401).
  keys:
    prefix: "ttv_"              # display/namespace prefix only; length & charset are fixed
    tls_terminated_upstream: true  # ATTESTATION (not a probe): you terminate TLS in front of
                                   # the data plane, so bearer keys don't cross cleartext HTTP
  limits:                       # HOT-reloadable
    - id: org_monthly           # a TEMPLATE limit — one counter per org
      attach:
        level: org              # global | org | team | user | key (global is the root scope)
      type: budget_usd          # budget_usd | rpm | tpm | quota_tokens | quota_requests | concurrency
      amount_usd: 1000          # decoded to nanodollars, echoed back canonically
      period: month             # day | week | month (UTC) for a ledger-period limit
      posture:
        mode: shadow            # the trust ladder again — watch would-refusals before enforcing

What each block does

  • mode — the trust ladder master switch. Start at ledger or shadow; a per-rule mode: can enforce one rule while the rest observes.
  • server — two planes on two ports. The data plane (listen) proxies traffic; the admin plane (admin) serves the dashboard and API. Keeping admin on loopback is what lets dashboard.auth.mode: off be safe.
  • upstreams — every provider is a named entry with a dialect and an env-var name for its key (never the key itself). See Providers for the built-in descriptors (Azure, Bedrock, Gemini, Vertex, …) and bring-your-own-key.
  • tiers — cheapest first. Routing rules point at tier names, so you can re-map a tier to a different model without touching a rule.
  • routing — ordered rules, a default_tier fail-safe, and a cooldown that diverts traffic off a failing upstream. Prefer the richer policy: form for anything branchier; see Policy language.
  • cache / governance — both are opt-in and default-off, and both start in mode: shadow here so you can watch would-hits and would-refusals before they bite. See Caching and Governance.

Grow from here

This config leaves several honest-by-default surfaces off. Add them when you need them, each opt-in and each documented in the configuration reference: