Governance

One team ships a retrieval feature that fans out five model calls per question. One agent gets stuck in a loop on a Saturday. One customer on your free tier works out that your gateway is a free gateway. In all three the money is gone before anyone looks, because the thing that notices is the invoice, and the invoice is monthly.

Governance moves that decision forward onto the request itself. A caller presents a virtual key, the key names a node in your tenancy tree, limits are attached to nodes, and those limits are evaluated before anything is sent upstream. A request over its cap comes back as a 429 naming the limit that stopped it, and it costs you nothing.

This page is the map: what you switch on, the one default that catches everybody, how a limit gets promoted from watching to enforcing, and which page owns each part. It assumes requests are already landing in your decision ledger and that you have met the trust ladder in core concepts. If you would rather do than read, Turn governance on is the how-to.

Nothing is governed until you ask for it

A fresh install runs no governance at all. The shipped example config carries the whole governance: block commented out, and an absent block is zero behaviour change: nothing is wired into the request path, no credential is looked up, no counter exists. Everything the rest of this section describes stays inert until you write the block.

Three steps switch it on.

governance:
  enabled: true # default false
  require_key: required # off (default) | optional | required
  tenants:
    - org: acme
      teams: [ml, search]
  limits:
    - id: org_monthly_budget
      attach: { level: org }
      type: budget_usd
      amount_usd: 1000
      period: month
      posture: { mode: shadow } # start here, not at the default

1. Enable the block. enabled: true is the switch that makes tokentriage validate read the governance block at all. With it absent or false, the validator returns before checking a single key inside it.
2. Mint keys against a node. tokentriage keys mint --owner org:acme/team:ml/user:dev1 prints a ttv_ secret once; the store keeps a hash of it, not the secret. The flag is --owner, not --node. Virtual keys covers scoping, rotation with a grace window, and revocation.
3. Attach limits to nodes. A limit either pins to one node: or declares a level: template that materialises at every node of that level. Six types ship: budget_usd, quota_tokens, quota_requests, rpm, tpm, and concurrency. The six limit types has each one’s keys and defaults, and the tenancy hierarchy has how a limit resolves along the chain from the key up to the global root.

The shape to avoid is declaring limits while leaving enabled at false. Governance arms on the presence of limits alone, so the gate runs over limits the validator did not open. tokentriage validate prints OK: configuration is valid, and the first thing to read those limits is the boot compiler, whose errors abort startup rather than warn.

Where a request meets governance

Governance is not one check, and knowing which of its moments stopped a request is most of debugging one. A request crosses it four times.

Read the left column downward as one request and the right column as what a caller can be told at each point. StageA runs before a byte of the body is read, so it knows the credential and the endpoint but not the model, which is why rate and concurrency caps are evaluated there. The body is then parsed and routing picks the model that will serve. StageB runs after that, probing spend and quota exhaustion and taking the tpm reserve. ServedCheck re-asserts the key’s model and provider scope against the model that will actually serve, and it runs before the response cache is consulted, so a warm cache cannot serve what a live call would have turned away. Settlement releases the concurrency lease and reconciles the reserve against the tokens the response used; it stops nothing, because the outcome is already decided.

So a row and the response the caller saw cannot disagree, and no path stops a request without writing a row. Every field, header and status that block can carry is on the refusal wire reference.

A limit is armed the moment you write it

Governance as a subsystem stays off until you write the block. An individual limit is the other way round, and this is the fact most worth carrying away from this page.

A limit you define with no posture: block at all is enforcing, hard, and fail-open from the moment it exists. The defaults are filled in when the limit resolves: an unspecified mode becomes on, an unspecified fail becomes open, and an enforcing limit with no enforce becomes hard. Nothing in your YAML says any of that, and there is no declared-but-idle state to land in by accident. Shadow is something you type.

Shadow first

Nobody should have to guess what a cap will do to their traffic before arming it, so every limit carries three modes and you climb them in order.

mode:	what happens on a request
off	the limit is skipped entirely and its counter is not touched
shadow	evaluated against live traffic; a breach is recorded as would_refuse and the request is served
on	a breach acts, per enforce:. hard stops the request; soft lets it through and writes nothing about the breach to the row

Shadow is a live evaluation, not a simulation. The limit reads the same counters, takes the same reserve and holds the same lease as it would while enforcing; the one thing it cannot do is stop a request. Here is the governance block of the row for the fourth request against a team_rpm limit capped at two per minute in shadow, served 200:

{
  "kid": "vk_FEKZdMs2UQRi",
  "node": "org:acme/team:ml/user:dev1",
  "verdict": "would_refuse",
  "would_refuse": ["team_rpm"],
  "user_asserted": false,
  "rotation_grace": false,
  "reserved_tokens": 16391,
  "serving_key": { "kid": "vk_FEKZdMs2UQRi", "node": "org:acme/team:ml/user:dev1" },
  "gate_us": { "a": 529, "b": 0 }
}

That capture is from a local sandbox pointed at a stand-in upstream, so the kid is that machine’s; the field shape is what your own rows carry. Two things to read off it. verdict takes one of two values, allow and would_refuse, and this row has the second, naming the limit in would_refuse. And reserved_tokens is the token reserve StageB took for that request: a tpm limit in shadow reserves the same estimate an enforcing one would, then hands it back at settlement.

There is a matching trap on the other side, and it costs people a query. A governance stop also writes verdict: "allow", because the stop is recorded in the sibling refused_by_limit block. Count refused_by_limit, not verdict.

Between shadow and on, tokentriage govern preview replays a proposed limit set over your recorded ledger and labels which of its own numbers were recomputed and which were approximated. Turn governance on walks the whole climb.

Fail open with disclosure

Some limits cannot answer their own question unaided. A budget_usd cap reads a counter that a separate accountant folds from ledger rows, and when that accountant is stale or dead the limit has nothing to compare against.

The default is to keep serving. The limit is suspended, traffic flows, and the suspension is disclosed on the request itself: the limit id appears in suspended on the decision row, and the response carries x-tt-coordination: suspended. The governance health endpoint’s suspensions list is a third surface, and it is the one that needs a shared-state tier — on a single-node instance that array stays empty whatever your limits do, and the row is where the disclosure lands. Availability wins, and you are told the guarantee was not in force for that request. fail: open selects this, and it is what you get when you write no fail: at all.

Fail-open is the rule and not the whole of it. Two paths do stop a request when the enforcement question cannot be answered, and both come back as a 503 enforcement_unavailable:

• require_key: required with no loadable key snapshot. A replica that cannot load its key store cannot tell a good credential from a bad one, so it declines rather than issuing a 401 it has no basis for. This one is not per-limit and not opt-in; a cold replica can serve it for the moments before its key store seeds.
• An uncheckable limit with fail: closed and enforce: hard. This one is opt-in and per limit. Pairing fail: closed with enforce: soft is a validation error, because a limit that only alerts has nothing to fail closed on.

Every other fault degrades to an allow with a disclosure. Neither 503 carries a Retry-After, since there is no time at which the retry is known to succeed.

What suspends, what converges, and what stays per-replica is governance across replicas; reading the health panel and clearing a suspension is operating governance.

Where to start

Four ways in, depending on what brought you here.

Get it running. Turn governance on is the how-to: off to shadow to enforcing on your own traffic, with govern preview between the rungs.

Understand the counting. How a limit counts is the page that makes a limit predictable. Four enforcement classes decide when a counter is written relative to the request, and that decides whether a cap can be crossed before it acts. A budget_usd counter folds off the request path, which is why a budget can overshoot.

Look something up. The six limit types has every type with its keys, defaults and failure mode. Refusal wire reference has every field, header and status a governance stop can carry.

Debug a 429. Why was this request refused? starts from the response a caller pasted into your issue tracker and works back to the limit, the node, and the change that fixes it.

What this section does not cover

Governance decides whether a request is allowed at all, and nothing else.

Which model serves it is routing, which runs between the two governance gates and cannot overrule a cap. What is inside the request or the response is guardrails, which inspect content rather than counting it. And what was spent, by whom, and attributed how is cost analytics.

The boundary with that last one is worth stating once, because budget_usd sits on it. This section teaches how a spend counter is built and what its accuracy bound is, because that is why a budget can be crossed before it acts. It does not teach you how to read a spend report. The inert knobs, the shapes that will not fire, and the numbers that want a second look are on what governance does not do.