# Governance Cap what a tenant, feature, or key can spend at the moment of the request instead of finding out on next month's invoice. Source: https://tokentriage.com/docs/build/governance --- import Callout from '../../../components/Callout.astro'; import NextSteps from '../../../components/NextSteps.astro'; import GovernanceGateDiagram from '../../../components/GovernanceGateDiagram.astro'; One team ships a retrieval feature that fans out five model calls per question. One agent gets stuck in a loop on a Saturday. One customer on your free tier works out that your gateway is a free gateway. In all three the money is gone before anyone looks, because the thing that notices is the invoice, and the invoice is monthly. Governance moves that decision forward onto the request itself. A caller presents a virtual key, the key names a node in your tenancy tree, limits are attached to nodes, and those limits are evaluated before anything is sent upstream. A request over its cap comes back as a 429 naming the limit that stopped it, and it costs you nothing. This page is the map: what you switch on, the one default that catches everybody, how a limit gets promoted from watching to enforcing, and which page owns each part. It assumes requests are already landing in your decision ledger and that you have met the trust ladder in [core concepts](/docs/concepts/core-concepts). If you would rather do than read, [Turn governance on](/docs/build/governance-rollout) is the how-to. ## Nothing is governed until you ask for it A fresh install runs no governance at all. The shipped example config carries the whole `governance:` block commented out, and an absent block is zero behaviour change: nothing is wired into the request path, no credential is looked up, no counter exists. Everything the rest of this section describes stays inert until you write the block. Three steps switch it on. ```yaml governance: enabled: true # default false require_key: required # off (default) | optional | required tenants: - org: acme teams: [ml, search] limits: - id: org_monthly_budget attach: { level: org } type: budget_usd amount_usd: 1000 period: month posture: { mode: shadow } # start here, not at the default ``` 1. **Enable the block.** `enabled: true` is the switch that makes `tokentriage validate` read the governance block at all. With it absent or `false`, the validator returns before checking a single key inside it. 2. **Mint keys against a node.** `tokentriage keys mint --owner org:acme/team:ml/user:dev1` prints a `ttv_` secret once; the store keeps a hash of it, not the secret. The flag is `--owner`, not `--node`. [Virtual keys](/docs/build/governance-keys) covers scoping, rotation with a grace window, and revocation. 3. **Attach limits to nodes.** A limit either pins to one `node:` or declares a `level:` template that materialises at every node of that level. Six types ship: `budget_usd`, `quota_tokens`, `quota_requests`, `rpm`, `tpm`, and `concurrency`. [The six limit types](/docs/build/governance-limits) has each one's keys and defaults, and [the tenancy hierarchy](/docs/build/governance-hierarchy) has how a limit resolves along the chain from the key up to the global root. The shape to avoid is declaring limits while leaving `enabled` at `false`. Governance arms on the presence of limits alone, so the gate runs over limits the validator did not open. `tokentriage validate` prints `OK: configuration is valid`, and the first thing to read those limits is the boot compiler, whose errors abort startup rather than warn. ## Where a request meets governance Governance is not one check, and knowing which of its moments stopped a request is most of debugging one. A request crosses it four times. Read the left column downward as one request and the right column as what a caller can be told at each point. StageA runs before a byte of the body is read, so it knows the credential and the endpoint but not the model, which is why rate and concurrency caps are evaluated there. The body is then parsed and routing picks the model that will serve. StageB runs after that, probing spend and quota exhaustion and taking the `tpm` reserve. ServedCheck re-asserts the key's model and provider scope against the model that will actually serve, and it runs before the response cache is consulted, so a warm cache cannot serve what a live call would have turned away. Settlement releases the concurrency lease and reconciles the reserve against the tokens the response used; it stops nothing, because the outcome is already decided. One function writes every governance stop a caller sees. It stamps the response headers, the response body, and the decision row's `refused_by_limit` block from one value, before any byte goes out. So a row and the response the caller saw cannot disagree, and no path stops a request without writing a row. Every field, header and status that block can carry is on [the refusal wire reference](/docs/build/governance-refusal-reference). ## A limit is armed the moment you write it Governance as a subsystem stays off until you write the block. An individual limit is the other way round, and this is the fact most worth carrying away from this page. A limit you define with no `posture:` block at all is enforcing, hard, and fail-open from the moment it exists. The defaults are filled in when the limit resolves: an unspecified `mode` becomes `on`, an unspecified `fail` becomes `open`, and an enforcing limit with no `enforce` becomes `hard`. Nothing in your YAML says any of that, and there is no declared-but-idle state to land in by accident. Shadow is something you type. A limit copied from an example with its `posture:` block trimmed off starts turning traffic away the first time it is crossed, and the first sign is a caller's 429. Write `posture: { mode: shadow }` on every new limit, read what it recorded, and promote it after. ## Shadow first Nobody should have to guess what a cap will do to their traffic before arming it, so every limit carries three modes and you climb them in order. | `mode:` | what happens on a request | |---|---| | `off` | the limit is skipped entirely and its counter is not touched | | `shadow` | evaluated against live traffic; a breach is recorded as `would_refuse` and the request is served | | `on` | a breach acts, per `enforce:`. `hard` stops the request; `soft` lets it through and writes nothing about the breach to the row | Shadow is a live evaluation, not a simulation. The limit reads the same counters, takes the same reserve and holds the same lease as it would while enforcing; the one thing it cannot do is stop a request. Here is the governance block of the row for the fourth request against a `team_rpm` limit capped at two per minute in shadow, served 200: ```json { "kid": "vk_FEKZdMs2UQRi", "node": "org:acme/team:ml/user:dev1", "verdict": "would_refuse", "would_refuse": ["team_rpm"], "user_asserted": false, "rotation_grace": false, "reserved_tokens": 16391, "serving_key": { "kid": "vk_FEKZdMs2UQRi", "node": "org:acme/team:ml/user:dev1" }, "gate_us": { "a": 529, "b": 0 } } ``` That capture is from a local sandbox pointed at a stand-in upstream, so the `kid` is that machine's; the field shape is what your own rows carry. Two things to read off it. `verdict` takes one of two values, `allow` and `would_refuse`, and this row has the second, naming the limit in `would_refuse`. And `reserved_tokens` is the token reserve StageB took for that request: a `tpm` limit in shadow reserves the same estimate an enforcing one would, then hands it back at settlement. There is a matching trap on the other side, and it costs people a query. A governance stop also writes `verdict: "allow"`, because the stop is recorded in the sibling `refused_by_limit` block. Count `refused_by_limit`, not `verdict`. Between shadow and `on`, `tokentriage govern preview` replays a proposed limit set over your recorded ledger and labels which of its own numbers were recomputed and which were approximated. [Turn governance on](/docs/build/governance-rollout) walks the whole climb. ## Fail open with disclosure Some limits cannot answer their own question unaided. A `budget_usd` cap reads a counter that a separate accountant folds from ledger rows, and when that accountant is stale or dead the limit has nothing to compare against. The default is to keep serving. The limit is suspended, traffic flows, and the suspension is disclosed on the request itself: the limit id appears in `suspended` on the decision row, and the response carries `x-tt-coordination: suspended`. The governance health endpoint's `suspensions` list is a third surface, and it is the one that needs a shared-state tier — on a single-node instance that array stays empty whatever your limits do, and the row is where the disclosure lands. Availability wins, and you are told the guarantee was not in force for that request. `fail: open` selects this, and it is what you get when you write no `fail:` at all. Fail-open is the rule and not the whole of it. Two paths do stop a request when the enforcement question cannot be answered, and both come back as a 503 `enforcement_unavailable`: - **`require_key: required` with no loadable key snapshot.** A replica that cannot load its key store cannot tell a good credential from a bad one, so it declines rather than issuing a 401 it has no basis for. This one is not per-limit and not opt-in; a cold replica can serve it for the moments before its key store seeds. - **An uncheckable limit with `fail: closed` and `enforce: hard`.** This one is opt-in and per limit. Pairing `fail: closed` with `enforce: soft` is a validation error, because a limit that only alerts has nothing to fail closed on. Every other fault degrades to an allow with a disclosure. Neither 503 carries a `Retry-After`, since there is no time at which the retry is known to succeed. What suspends, what converges, and what stays per-replica is [governance across replicas](/docs/build/governance-distributed); reading the health panel and clearing a suspension is [operating governance](/docs/build/governance-operations). ## Where to start Four ways in, depending on what brought you here. **Get it running.** [Turn governance on](/docs/build/governance-rollout) is the how-to: off to shadow to enforcing on your own traffic, with `govern preview` between the rungs. **Understand the counting.** [How a limit counts](/docs/build/governance-enforcement-classes) is the page that makes a limit predictable. Four enforcement classes decide when a counter is written relative to the request, and that decides whether a cap can be crossed before it acts. A `budget_usd` counter folds off the request path, which is why a budget can overshoot. **Look something up.** [The six limit types](/docs/build/governance-limits) has every type with its keys, defaults and failure mode. [Refusal wire reference](/docs/build/governance-refusal-reference) has every field, header and status a governance stop can carry. **Debug a 429.** [Why was this request refused?](/docs/build/governance-refusals) starts from the response a caller pasted into your issue tracker and works back to the limit, the node, and the change that fixes it. ## What this section does not cover Governance decides whether a request is allowed at all, and nothing else. Which model serves it is [routing](/docs/build/routing), which runs between the two governance gates and cannot overrule a cap. What is inside the request or the response is [guardrails](/docs/guardrails/overview), which inspect content rather than counting it. And what was spent, by whom, and attributed how is [cost analytics](/docs/build/cost-analytics). The boundary with that last one is worth stating once, because `budget_usd` sits on it. This section teaches how a spend counter is built and what its accuracy bound is, because that is why a budget can be crossed before it acts. It does not teach you how to read a spend report. The inert knobs, the shapes that will not fire, and the numbers that want a second look are on [what governance does not do](/docs/build/governance-gaps).