# The six limit types Every governance limit type with its keys, defaults, cost, failure mode, and how to tell it is working. Source: https://tokentriage.com/docs/build/governance-limits --- import Callout from '../../../components/Callout.astro'; import Details from '../../../components/Details.astro'; import NextSteps from '../../../components/NextSteps.astro'; Six limit types ship, and choosing between them is mostly a question about timing. Do you want a cap on what a team spends this month, or on how fast it can arrive this minute? Those two answers are counted in different places. A monthly dollar cap is settled from your ledger after each request finishes, so it can drift past its number. A per-minute cap is counted at the door, before anything is forwarded, so it holds. Each entry carries the config, the defaults, the cost, the failure mode, and the output that shows it working, in the order the product registers them. If you have not met the four counting mechanisms, read [how a limit counts](/docs/build/governance-enforcement-classes) first. This page names each type's class and links there. ## What each type caps | Type | Caps | Class | Unit | `error.type` | Status | |---|---|---|---|---|---| | `budget_usd` | dollars spent in a calendar period | `LedgerSettled` | `usd` | `budget_exceeded` | 429 | | `quota_tokens` | input plus output tokens in a calendar period | `LedgerSettled` | `tokens` | `quota_exhausted` | 429 | | `quota_requests` | forwarded requests in a calendar period | `LedgerSettled` | `requests` | `quota_exhausted` | 429 | | `rpm` | requests admitted per window | `AdmissionCounted` | `requests` | `rate_limited` | 429 | | `tpm` | tokens reserved per window | `ReserveReconcile` | `tokens` | `rate_limited` | 429 | | `concurrency` | simultaneous in-flight requests | `InflightLease` | `concurrent-requests` | `concurrency_exceeded` | 429 | Every limit breach is a 429. One further 429 exists: `quota_cost_unfittable`, when a single request asks for more than the window's burst can ever hold. Only `tpm` reaches it in this build, because an `rpm` probe costs one unit and a burst is at least one. Anything turned away for a reason other than a limit breach carries a different status, and [the refusal wire reference](/docs/build/governance-refusal-reference) has them all. `error.type` names the family, and `refused_by.limit_type` names the type that fired, so `rpm` and `tpm` share `rate_limited` and stay distinguishable on the row. ## Keys every type shares These keys mean the same thing everywhere, so the per-type tables that follow list only what is specific to that type. | Key | Default | What it does | |---|---|---| | `id` | required | names the limit on every 429 and every row; charset `[a-z0-9_]` | | `attach.node` or `attach.level` | required, one of the two | pins the limit to one node, or templates it at `org`, `team` or `user` | | `match.model`, `.provider`, `.endpoint` | none | narrows which requests the limit counts | | `posture.mode` | `on` | `shadow` records a would-refuse and admits; `off` is inert | | `posture.enforce` | `hard` | `soft` admits on breach instead of returning a 429 | | `posture.fail` | `open` | `closed` with `enforce: hard` turns an uncheckable limit into a 503 | | `posture.thresholds` | none | alert fractions in `[0,1]` | | `override.allow` | `false` | permits a per-node override, on three of the six types | | `override.max_amount`, `override.max_amount_usd` | none | the ceiling an override may not pass; omitting it declares no ceiling at all | A limit with no `posture:` block is hard-enforcing and fail-open. Shadow is something you type, not something you get by leaving the block out. Two of these keys have edges. `match.model` and `match.provider` read values the gate has not parsed yet on `rpm` and `concurrency`, so those two types can be narrowed only by `match.endpoint`. And `attach.level` also accepts `global` and `key`, neither of which instantiates a node in this build; a fleet-wide limit is written `attach: { node: __global }`. [What governance does not do](/docs/build/governance-gaps) collects both, and the other keys that load cleanly and change nothing. ## The six types ### `budget_usd` Caps the dollars a scope may spend inside a calendar period, folded from priced ledger rows. ```yaml - id: org_monthly_budget attach: { level: org } type: budget_usd amount_usd: 1000 # dollars, quantized to the nanodollar grid period: month # day | week | month posture: { mode: "on", enforce: hard, fail: open } override: { allow: true, max_amount_usd: 5000 } ``` **How it counts.** A worker folds each settled row into a per-scope counter after the request has finished, and admission reads a snapshot of that counter. A priced row adds its cost. An `UNPRICED` row adds nothing to the dollar total and is still counted as a row. A cache replay adds a priced zero. That is the [`LedgerSettled`](/docs/build/governance-enforcement-classes#ledgersettled--spend-and-quota) class. **Knobs.** | Key | Default | What it does | |---|---|---| | `amount_usd` | required | the cap in dollars; `amount` on this type is a config error | | `period` | required | `day`, `week` or `month`; a rate window is a config error | | `algorithm`, `store` | not accepted | either one is a config error on a spend or quota type | | `burst` | not read | accepted by validation and consumed by nothing | **What it costs.** One snapshot read on the request path. Off the path, one counter per limit and scope key, fed from a bounded queue that drops and counts rows rather than slowing the proxy. **How it fails.** It overshoots. The counter is written after the request that should have filled it, so spend already in flight is missing from the number admission reads. Here is the `refused_by` block of a captured 429: ```json {"limit_id":"org_monthly_budget","limit_type":"budget_usd","unit":"usd","node":"org:acme","period":"month","resets_at":"2026-09-01T00:00:00Z","retry_after_s":1912510,"observed":0.0000126,"limit":0.000001} ``` `observed` is 12.6 times the cap. That capture used a limit of one millionth of a dollar so a single request would pass it, which exaggerates the ratio; the mechanism is what generalises. When the accountant is stale or has not armed, the limit becomes uncheckable and the request is admitted with `suspended:["org_monthly_budget"]` on its row, unless the limit carries `fail: closed` with `enforce: hard`, which produces a 503 instead. **How to tell it is working.** `GET /tt/v1/me` with the caller's key returns each armed limit and its live numbers. Same sandbox, so the same millionth-of-a-dollar cap: ```json {"limit_id":"org_monthly_budget","type":"budget_usd","class":"spend","unit":"usd","node":"org:acme","period":"month","resets_at":"2026-09-01T00:00:00Z","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"limit_usd":{"usd":0.000001,"estimated":false,"unpriced":false},"observed_usd":{"usd":0.0000126,"estimated":true,"unpriced":false},"remaining_usd":{"usd":0,"estimated":true,"unpriced":false}} ``` **Reach for something else when** you need a cap that cannot be passed. A dollar figure is only known once the provider answers, so no spend limit can be a hard stop. `rpm` and `concurrency` are counted before the request leaves, and they hold. ### `quota_tokens` Caps the tokens a scope may consume inside a calendar period, counting input and output together. ```yaml - id: team_monthly_tokens attach: { level: team } type: quota_tokens amount: 50000000 # total tokens, input + output period: month posture: { mode: "on", enforce: hard, fail: open } ``` **How it counts.** The same off-path fold as `budget_usd`, metered in tokens. The whole token term is one line: ```text c.AddTokens(rec.Usage.InputTokens, rec.Usage.OutputTokens) ``` There is no `metric` key in this build, so input-only and output-only caps are not expressible. Two comments elsewhere in the source advertise `metric: input|output|total`; the type's own header states total-only, and no such field exists on the config struct or the limit definition. [`LedgerSettled`](/docs/build/governance-enforcement-classes#ledgersettled--spend-and-quota) class. **Knobs.** | Key | Default | What it does | |---|---|---| | `amount` | required | the cap as a token count; `amount_usd` on this type is a config error | | `period` | required | `day`, `week` or `month` | | `algorithm`, `store`, `burst` | not accepted or not read | two config errors and one key nothing reads | **What it costs.** Nothing on the request path. Each limit keeps its own counter per scope key, and each settled row is folded once per selected instance. **How it fails.** It lags the same way `budget_usd` does. Two details differ. An `UNPRICED` row still consumes tokens, because only `budget_usd` excludes unpriced work from its total. A cache replay consumes zero tokens and still counts as a row. `quota_tokens` sums the row's input and output tokens; `tpm` settles against the provider-reported total, a separate field that can include reasoning tokens. The two can disagree about the same request. **How to tell it is working.** The `EXACT` block of [`govern preview`](#compiling-the-set-before-it-boots) recomputes it from ledger history. Its line for this limit, over the five-row sandbox ledger used later on this page: ```text [quota_tokens] team_monthly_tokens @ org:acme/team:ml observed 60 / ceiling 50000000 (frac 0.0000, rows 1) ``` **Reach for something else when** you care about money rather than volume. Token counts across models are not comparable spend, and `budget_usd` prices each row through the catalog. ### `quota_requests` Caps forwarded requests in a calendar period. Its entire fold is one row increment. ```yaml - id: team_daily_requests attach: { level: team } type: quota_requests amount: 20000 # forwarded requests period: day posture: { mode: "on", enforce: hard, fail: open } ``` **How it counts.** Every forwarded row counts once, whether priced, `UNPRICED`, or served from cache: ```text func (l *compiledQuotaRequests) Fold(rec *declog.Record, c *govern.Counters) { c.AddRow(rec.Cost.Unpriced, rec.Cost.Estimated) } ``` A row for a request that was turned away is skipped before the fold, so a storm of 429s consumes no quota. [`LedgerSettled`](/docs/build/governance-enforcement-classes#ledgersettled--spend-and-quota) class. **Knobs.** | Key | Default | What it does | |---|---|---| | `amount` | required | the cap as a request count | | `period` | required | `day`, `week` or `month` | | `algorithm`, `store`, `burst` | not accepted or not read | two config errors and one key nothing reads | **What it costs.** Nothing on the request path, and one integer per scope key off it. **How it fails.** It lags, and a cache hit consumes a slot. A caller served entirely from your cache spends no money and still spends quota. **How to tell it is working.** The same `EXACT` block: ```text [quota_requests] team_daily_requests @ org:acme/team:ml observed 1 / ceiling 20000 (frac 0.0001, rows 1) ``` **Reach for something else when** you want to pace arrivals rather than cap a period's volume. That is `rpm`. ### `rpm` Caps requests admitted per rolling window, counted at the gate before the body is read. `rps` is this type with `period: 1s`, not a separate registration. ```yaml - id: team_rpm attach: { level: team } type: rpm amount: 300 # requests per window period: 1m # a window duration, not a calendar period posture: { mode: shadow } ``` **How it counts.** One algorithm step per request through a counter store, taken at the first gate. The number is the admission decision, so it cannot lag. The step is one-way: a probe that admitted stays consumed even when a later limit stops the same request. [`AdmissionCounted`](/docs/build/governance-enforcement-classes#admissioncounted--rpm) class. **Knobs.** | Key | Default | What it does | |---|---|---| | `amount` | required | requests admitted per window | | `period` | required | any Go duration, such as `1s` or `1m`; a calendar period is a config error | | `burst` | `amount` | how many arrivals a drained key absorbs at once | | `algorithm` | `gcra` | also `fixed_window`, `sliding_window`, `token_bucket` | | `store` | `local` | also `redis`, which needs `shared_state.redis`, and `share` | Burst defaults to the full amount, so a drained key admits `amount` arrivals at once and then one per emission interval. [Rate algorithms](/docs/build/governance-rate-algorithms) covers what each choice guarantees, the window algorithms' 65,535 ceiling, and the `gcra` requirement on `store: share`. **What it costs.** With the default `local` store: a read lock on one of 256 shards to find the key's cell, then a lock-free compare-and-swap loop over a single int64. Memory is one cell per scope key, reclaimed by a collector that runs only when a shard gains a new key. With `store: redis`, one scripted round trip on the request path under a 5 ms deadline. **How it fails.** Two ways. A limit with no `period:` passes `tokentriage validate` and then stops the boot: ```text Error: governance: compile limits: govern: limit "team_rpm": rpm requires a rate-class window period (e.g. 1s, 1m), not a calendar period ``` Both governance example configs in the product repo have that shape. Across replicas with the default store, each replica keeps its own cell, so a fleet of N admits up to N times the amount. That bound is published rather than hidden; [governance across replicas](/docs/build/governance-distributed) has the arithmetic. **How to tell it is working.** With `governance.keys.disclose_limits` at its default of `true`, a served response advertises the tightest rate limit: ```console HTTP/1.1 200 OK Content-Type: application/json Ratelimit-Policy: "team_rpm";q=300;w=60 X-Ratelimit-Limit-Requests: 300 X-Ratelimit-Remaining-Requests: 59 X-Tt-Request-Id: tt_tbuhetygwddxiaprkw3j3fkwze ``` That capture came from a limit configured with `burst: 60`, and `Remaining` tracks the burst depth rather than the 300 per minute. The limit was in `mode: shadow` and still advertised its policy. **Reach for something else when** request cost varies by orders of magnitude. Three hundred one-token requests and three hundred 100,000-token requests look identical to `rpm`. `tpm` prices the window in tokens. ### `tpm` Caps tokens per rolling window by reserving an estimate at the gate and settling it to observed usage afterwards. ```yaml - id: team_tpm attach: { level: team } type: tpm amount: 200000 # tokens per window period: 1m match: { model: "gpt-*" } posture: { mode: "on", enforce: hard, fail: open } ``` **How it counts.** After the body is parsed, the gate reserves `est_input + min(max_tokens, 16384)` tokens, where `est_input` is the prompt byte count divided by four. With no `max_tokens` on the request, the reserve is `est_input + 16384`. At settlement the difference between the reserve and the observed total is committed as a signed adjustment, with no second admission check. [`ReserveReconcile`](/docs/build/governance-enforcement-classes#reservereconcile--tpm) class. **Knobs.** | Key | Default | What it does | |---|---|---| | `amount` | required | tokens per window | | `period` | required | any Go duration; a calendar period is a config error | | `burst` | `amount` | must be at least 16384, the reserve cap | | `algorithm` | `gcra` | the only value that compiles | | `store` | `local` | also `share`; `redis` is turned down at compile | **What it costs.** One write to the cell at the gate and, unless the estimate matched observed usage, a second at settlement, plus one reservation held for the life of the request. Every `tpm` 429 is flagged `approximate: true`, because a reserve is an estimate. **How it fails.** Three ways. A burst under 16,384 does not compile, and burst defaults to amount, so a limit smaller than the reserve cap cannot be built at all: ```text Error: governance: compile limits: govern: limit "team_tpm": tpm limit "team_tpm": burst 500 is below the reserve cap 16384 — an ordinary max-token reserve could never fit (a permanent 100% refusal); raise burst (default is amount 500) to at least the reserve cap ``` The guard reads the effective burst, so `type: tpm` with `amount: 10000` and no `burst:` stops the same way. Raise `amount` to 16,384 or more, or set `burst: 16384` and keep the smaller rate. Second, a request whose reserve exceeds the burst can fit at no future moment, and it gets `quota_cost_unfittable` with no useful retry hint. Third, settlement commits a debit past the cap. An under-estimate is corrected after the fact, so the window can go over by the size of the miss. **How to tell it is working.** A served response advertises the token pair: ```console Ratelimit-Policy: "team_tpm";q=20000;w=60 X-Ratelimit-Limit-Tokens: 20000 X-Ratelimit-Remaining-Tokens: 3609 ``` That capture came from a limit configured with `amount: 20000`, and `20000 − 16391 = 3609` is one request holding a 16,391-token reserve. **Reach for something else when** you want a calendar-period token cap rather than a per-minute one; that is `quota_tokens`. Also when a fleet total has to be exact, since `tpm` cannot bind `store: redis` and an `rpm` limit can. ### `concurrency` Caps how many requests a scope may have in flight at the same instant. ```yaml - id: user_concurrency attach: { level: user } type: concurrency amount: 8 # simultaneous in-flight requests posture: { mode: "on", enforce: hard, fail: open } ``` **How it counts.** A live integer per scope key, incremented at the first gate and decremented once when the request finishes. No window, no algorithm, no counter store. The release is idempotent, so a double release cannot free two slots, and settlement runs on every terminal path including a client disconnect. [`InflightLease`](/docs/build/governance-enforcement-classes#inflightlease--concurrency) class. **Knobs.** | Key | Default | What it does | |---|---|---| | `amount` | required | maximum simultaneous in-flight requests per scope key | | `store` | `local` | also `share`, which caps this replica at `ceil(amount / N_live)` | | `algorithm` | not accepted | a config error on this type | | `burst` | not read | accepted by validation and consumed by nothing | | `period` | not read | a window value loads and changes nothing at the gate; a calendar value is a config error | A window `period:` here is worse than inert. The period is one of the components a counter's scope key is built from, so adding or removing a `period:` that no code reads still splits the live gauge into two. **What it costs.** A compare-and-swap on one int64 gauge at the gate and one atomic decrement at settlement. Memory is one gauge per scope key, held for the life of the process rather than reclaimed. **How it fails.** The gauge is per replica and lives in memory, so a restart drops it to zero and a rolling restart briefly admits more than the cap. And the 429 body reports `observed: 0` beside a live `limit`. This capture is from a sandbox limit at `amount: 2` with four requests fired at once against a 3-second upstream: ```json {"error":{"type":"concurrency_exceeded","message":"request refused by limit user_concurrency","refused_by":{"limit_id":"user_concurrency","limit_type":"concurrency","unit":"concurrent-requests","node":"org:acme/team:ml/user:dev1","observed":0,"limit":2},"enforcement":{"posture":"enforcing","approximate":false}}} ``` Two requests were in flight at that moment, so treat `observed` on this type as unpopulated and read the gauge from `/tt/v1/me` instead. There is also no `Retry-After` worth acting on, because a slot frees when a request finishes rather than on a clock. **How to tell it is working.** The `/tt/v1/me` row carries the live gauge and the cap this replica is enforcing: ```json {"limit_id":"user_concurrency","type":"concurrency","class":"count","unit":"concurrent-requests","node":"org:acme/team:ml/user:dev1","period":"","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"amount":10,"observed":0,"remaining":10} ``` That capture came from a limit configured with `amount: 10` and no traffic in flight. **Reach for something else when** the problem is arrival rate rather than simultaneity. One slow client holding two connections and one fast client making 600 calls a minute are different problems, and `rpm` addresses the second. ## Compiling the set before it boots Most of the guards named in this page fire at compile, and compile happens at boot. You want that answer before a deploy, not during one. `tokentriage validate` does not compile limits. It checks the config surface: names, units, period against class, store against class. Every per-type guard on this page runs later, at boot or hot reload, and a config that trips one stops the process. Run `tokentriage govern preview -c --log ` first. It runs the same compile and prints the same text behind its own `govern preview:` prefix, and it errors if the decision log does not exist yet. Preview also recomputes each limit over your ledger, which is the fastest way to see all six armed at once. Every armed limit gets a line of its own under one of three labels, and the run below is cut back to one label and one of its lines, with `# …` marking what was dropped; [turn governance on](/docs/build/governance-rollout#preview-the-cap-against-traffic-you-already-have) prints a report in full. ```console $ tokentriage govern preview -c sixlimits.yaml --log ./gov/state2/decisions.jsonl # … EXACT — LedgerSettled recomputation (hand-count-verifiable, exact): [quota_requests] team_daily_requests @ org:acme/team:ml observed 1 / ceiling 20000 (frac 0.0001, rows 1) # … ``` That ran against the six configs on this page and a five-row sandbox ledger, so read the labels rather than the figures.
Each of these loads clean under `tokentriage validate` and stops the compile: - `rpm` or `tpm` with no `period:` at all - `tpm` with `algorithm: sliding_window`, or any value other than `gcra` - `tpm` with an effective burst under 16,384 - `tpm` with `store: redis` alongside a `shared_state.redis` block, which reports `type "tpm" (reserve/reconcile) cannot bind store "redis"` - `rpm` with `algorithm: fixed_window` or `sliding_window` and an amount above 65,535 One class of error is caught by neither command. Preview resolves a store *name* without building the store, so a bad algorithm-to-store pairing — `store: share` on any algorithm other than `gcra`, or `token_bucket` naming `share` or `redis` — surfaces only at boot. The `tpm` and `redis` row above is different: that one is a store-*class* rule the compiler checks, which is why preview reports it. `validate` does catch six shapes on its own: `store:` on a spend or quota type, `algorithm:` on a spend or quota type, `algorithm:` on `concurrency`, `store: redis` on `concurrency`, a calendar `period:` on a rate type, and `store: redis` on `rpm` or `tpm` with no `shared_state.redis` block.
## Per-node overrides reach three of the six A template gives every node at its level the same cap. When one node needs a different number, whether you can give it depends on the type. A per-node override may raise or lower `budget_usd`, `quota_tokens` and `quota_requests`. An override on `rpm`, `tpm` or `concurrency` is turned down, because those three bake the amount into a window store or a gauge when they compile and do not read a per-node value. `GET /tt/api/v1/governance/limits` states that rule as data: every armed definition carries `override_allowed`. This response is from a sandbox config with a tiny shadow `rpm`, not from the samples above, so read the flags rather than the amounts: ```json { "definitions": [ {"limit_id":"org_monthly_budget","limit_type":"budget_usd","class":"spend","unit":"usd","period":"month","posture":"enforcing","attach":"template","level":"org","amount":{"usd":1000,"estimated":false,"unpriced":false},"override_allowed":true,"override_max":{"usd":5000,"estimated":false,"unpriced":false}}, {"limit_id":"team_rpm","limit_type":"rpm","class":"count","unit":"requests","period":"1m0s","posture":"alert_only","attach":"template","level":"team","amount_count":2,"override_allowed":false}, {"limit_id":"team_tpm","limit_type":"tpm","class":"count","unit":"tokens","period":"1m0s","posture":"enforcing","attach":"template","level":"team","amount_count":20000,"override_allowed":false}, {"limit_id":"user_concurrency","limit_type":"concurrency","class":"count","unit":"concurrent-requests","posture":"enforcing","attach":"template","level":"user","amount_count":10,"override_allowed":false} ], "overrides": [], "note": "definitions are the ARMED limits enforced now; overrides are persisted runtime writes that take effect on the NEXT governance reload (compile-bound). No override is reported as live-enforced here." } ``` An override attempted on one of the other three is dropped with a reason naming the type: ```text limit type "rpm" does not support per-node overrides (its cap is enforced by a per-window/gauge store that cannot honor a per-node amount); overrides apply only to spend/quota limits (budget_usd, quota_tokens, quota_requests) ``` `override.max_amount` and `override.max_amount_usd` declare a ceiling for the template rather than a per-node instance, and omitting the ceiling while setting `allow: true` leaves any positive amount acceptable. [The tenancy hierarchy](/docs/build/governance-hierarchy) owns the rest of the mechanism. ## What this page does not cover The types, not the machinery around them. What each class implies under burst and across replicas is [how a limit counts](/docs/build/governance-enforcement-classes). What each algorithm guarantees is [rate algorithms](/docs/build/governance-rate-algorithms). Which instances a request selects is [the tenancy hierarchy](/docs/build/governance-hierarchy). Every field on a 429 is [the refusal wire reference](/docs/build/governance-refusal-reference). What was spent, by whom, and how it was attributed is [cost analytics](/docs/build/cost-analytics).