Governance, virtual keys & budgets — TokenTriage Docs

Governance mints virtual keys and enforces per-tenant budgets, quotas, rate, and concurrency limits — and it is built so that a refusal can never lie. It is off by default: an absent governance: block is zero behavior change.

Virtual keys

Virtual keys are ttv_-prefixed bearer tokens bound to a tenancy node, with an optional scope (models / providers / endpoints). They are stored as SHA-256 hashes, never a stored secret (shown once at mint), and the ttv_ prefix is stripped before the upstream forward. Read-only tenant share tokens use the tts_ prefix; a caller introspects its own limits at GET /tt/v1/me.

Minting keys

$ tokentriage keys mint --owner org:acme/team:ml/user:dev1 --label dev1-key --expiry 2160h  # 90 days
kid: vk_…   secret: ttv_…   (shown once)

keys mint also takes --models, --providers, --endpoints (comma-separated scopes; empty = unrestricted) and --allow-passthrough. Manage the lifecycle with keys list, keys revoke <KID> (terminal), and keys rotate <KID> --grace <dur>.

The six limit types

Six limit types run over one frozen engine:

Type	Caps	Breach → status
`budget_usd`	USD spend (dollars-decimal, stored on the nanodollar grid)	`budget_exceeded` → 429
`quota_tokens`	Cumulative token count over a period	`quota_exhausted` → 429
`quota_requests`	Cumulative forwarded-request count over a period	`quota_exhausted` → 429
`rpm`	Requests-per-minute rate (GCRA)	`rate_limited` → 429
`tpm`	Tokens-per-minute rate (reserve → reconcile)	`rate_limited` → 429
`concurrency`	Simultaneous in-flight requests	`concurrency_exceeded` → 429

(Other refusal reasons: invalid_key → 401, scope_violation → 403, enforcement_unavailable → 503.)

The hierarchy

Limits attach to an org / team / user / key tenancy tree. A limit either pins to a node or declares a level template that materializes per node at that level. Resolution walks the principal’s chain ∪ GLOBAL, deepest-first, as a conjunction: a parent cap binds every child, and the tightest applicable limit wins. Template limits take per-node overrides bounded by a ceiling — but overrides apply only to the spend/quota classes; an override on rpm/tpm/concurrency is rejected. A cross-scope request is refused 403 scope_violation, naming only the violated dimension.

Tenant governance: an org/team/key hierarchy with per-node limits and template overrides. — Tenant governance — template limits with per-node overrides across the hierarchy, all disclosed in the ledger.

Row == wire

The one deliberate asymmetry: a 401 discloses less on the wire than on the row (no refused_by — no existence oracle), by design.

Fail-open with disclosure

If the accountant behind a spend or quota limit is stale or dead, that limit is suspended (traffic flows) and the row plus the health panel disclose suspended:[<limit_id>] — never a silent block, never a fabricated 429. The guarantee is uncheckable, so it is disclosed as uncheckable rather than enforced on a guess.

Shadow-first

Every limit carries a posture on the enablement ladder — you never enforce blind. It climbs mode: off → mode: shadow (evaluated; records a would_refuse; never refuses) → mode: on with enforce: soft (alert-only) or enforce: hard (refuses, per fail:). Dry-run a proposed limit set over your logs with tokentriage govern preview.

governance:
  enabled: true
  require_key: required               # anonymous traffic gets an honest 401
  limits:
    - id: org_monthly_budget
      attach: { level: org }
      type: budget_usd
      amount_usd: 1000
      period: month
      posture: { mode: shadow }        # observe would-refusals first, then promote to on

Upstream credentials & honest attribution

A virtual key never maps to a provider credential value — it maps to a tenancy node (org / team / user). Upstream credentials resolve per-upstream, from the environment-variable names each upstream references (upstreams[].api_key_env), selected by routing — not by the tenancy node. The value never enters config, store, or ledger row, and the inbound ttv_ secret is stripped before the request leaves for the provider, so a virtual-key secret is never forwarded upstream.

What keeps attribution honest is the serving-key disclosure on every decision row: decision.governance.serving_key records the kid, the tenancy node, and any rotation lineage (rotated_from) of the virtual key that served the row — so “which key, which tenant” is always answerable from the ledger, wire-faithfully.

Across replicas

An opt-in distributed tier (shared_state.redis/postgres, each requiring external: allow) coordinates governance fleet-wide — spend converges, token/request quotas coordinate, and multi-replica key revocation is real; tpm stays per-replica by design (its signed reserve→reconcile is refused at validate for a shared store). The single-node default dials nothing. See Multi-replica & scale-out for the disclosed convergence bounds.

Distributed governance health: shared-state reachability, the suspended-limit list, and the widened convergence bound. — Governance health — shared-state reachability, suspended limits, and the disclosed convergence bound when coordination degrades.

Two CI-validated example configs live in examples/governance/. Related: Security & secrets · Migrate from LiteLLM.