# How a limit counts The four enforcement classes, and how to predict any limit under burst, overshoot, and more than one replica Source: https://tokentriage.com/docs/build/governance-enforcement-classes --- import Callout from '../../../components/Callout.astro'; import Details from '../../../components/Details.astro'; import NextSteps from '../../../components/NextSteps.astro'; import GovernanceClassesDiagram from '../../../components/GovernanceClassesDiagram.astro'; You set two limits. One caps spend in dollars, the other caps requests per minute. Both are enforcing, and both are a number in the same config block. The rate limit stops the 301st request in a minute and not the 300th. The spend cap, in the capture below, had already recorded 12.6 times its own figure by the time it stopped anything. That gap is not a tuning problem and it is not a defect. The two limits belong to different enforcement classes, and a class is one promise: when the counter is written, relative to the request being counted. Three of the four classes write at the admission gate. The fourth writes afterwards, off the request path, and the distance between those two moments is the overshoot. Four classes cover all six limit types, and every type belongs to one of them. Once you know a limit's class you can predict what a burst does to it, whether its cap can be passed, what it costs per request, what happens when you run more than one replica, and what its 429 tells the caller. [The six limit types](/docs/build/governance-limits) is the per-type reference; this page is the model underneath it. ## The four refusals, side by side The classes are not an internal taxonomy you have to take on trust. Every governance 429 carries an `enforcement` block, and that block is shaped by the class. Here is a `budget_usd` cap of `amount_usd: 0.000001`, which one request is enough to blow through. ```console $ curl -i -X POST http://127.0.0.1:8790/v1/chat/completions \ -H 'content-type: application/json' \ -H "Authorization: Bearer $TT_KEY" \ -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"What is the capital of Germany?"}]}' HTTP/1.1 429 Too Many Requests Content-Type: application/json Ratelimit: "org_monthly_budget";r=0;t=1912510 Retry-After: 1912510 X-Tt-Refusal-Type: budget_exceeded X-Tt-Refused-By: org_monthly_budget X-Tt-Request-Id: tt_i3f2naenug5aanctu6pcjrofcy {"error":{"type":"budget_exceeded","message":"request refused by limit org_monthly_budget","refused_by":{"limit_id":"org_monthly_budget","limit_type":"budget_usd","unit":"usd","node":"org:acme","period":"month","resets_at":"2026-09-01T00:00:00Z","retry_after_s":1912510,"observed":0.0000126,"limit":0.000001},"enforcement":{"posture":"enforcing","approximate":true,"bound":{"tee_lag_rows":0,"dropped_rows":0},"note":"spend from in-flight and streaming requests admitted before this refusal may not yet be reflected"}}} ``` That capture comes from a source build against a local sandbox with a stub OpenAI-compatible upstream, so the port, the request id and the key are ours; yours differ. The dollar figures are arithmetic over the stub's token counts rather than anyone's bill. Read `observed` against `limit`: **0.0000126 against 0.000001**, so the cap was 12.6 times behind when it stopped the request. `retry_after_s: 1912510` is 22 days, because a calendar budget resets on the next month boundary and not on a rolling window. Three other limits, driven against the same sandbox, produce three different `enforcement` blocks. | limit type | class | `approximate` | `bound` | `note` | |---|---|---|---|---| | `budget_usd` | LedgerSettled | `true` | `{"tee_lag_rows":0,"dropped_rows":0}` | present, and it names the lag | | `rpm` | AdmissionCounted | `false` | absent | absent | | `tpm` | ReserveReconcile | `true` | absent | absent | | `concurrency` | InflightLease | `false` | absent | absent | Two of them claim precision and two admit an approximation, and the two approximations are not the same shape. `budget_usd` carries a bound object and a sentence explaining what is missing from its number. `tpm` says `approximate: true` and stops there, because a reserve is an estimate but nothing is folding behind it. Four classes, four disclosures, and the wire tells you which is which without you reading any config.
`rpm`, AdmissionCounted: ```console HTTP/1.1 429 Too Many Requests Content-Type: application/json Ratelimit: "team_rpm";r=0;t=20 Retry-After: 20 X-Tt-Refusal-Type: rate_limited X-Tt-Refused-By: team_rpm X-Tt-Request-Id: tt_e6gbpncee32dbu7y5jpsfr4ax4 {"error":{"type":"rate_limited","message":"request refused by limit team_rpm","refused_by":{"limit_id":"team_rpm","limit_type":"rpm","unit":"requests","node":"org:acme/team:ml","period":"1m0s","retry_after_s":20},"enforcement":{"posture":"enforcing","approximate":false}}} ``` `tpm`, ReserveReconcile. Same `error.type` as `rpm`, because the type names the family and `limit_type` names the limit: ```console HTTP/1.1 429 Too Many Requests Content-Type: application/json Ratelimit: "team_tpm";r=0;t=39 Retry-After: 39 X-Tt-Refusal-Type: rate_limited X-Tt-Refused-By: team_tpm {"error":{"type":"rate_limited","message":"request refused by limit team_tpm","refused_by":{"limit_id":"team_tpm","limit_type":"tpm","unit":"tokens","node":"org:acme/team:ml","period":"1m0s","retry_after_s":39},"enforcement":{"posture":"enforcing","approximate":true}}} ``` `concurrency`, InflightLease. Four requests fired at once against `amount: 2`: ```console {"error":{"type":"concurrency_exceeded","message":"request refused by limit user_concurrency","refused_by":{"limit_id":"user_concurrency","limit_type":"concurrency","unit":"concurrent-requests","node":"org:acme/team:ml/user:dev1","observed":0,"limit":2},"enforcement":{"posture":"enforcing","approximate":false}}} ```
## When the counter is written Those four disclosures come from one difference, and it is a difference in timing rather than in arithmetic. The diagram puts all four classes on a single request's timeline so you can see where each one touches its counter. Read the diagram left to right along one request: the admission gate, the forwarded call, the response, settlement. The three upper rows all touch their counter at the gate. `rpm` counts the arrival and there is nothing to hand back. `concurrency` takes a lease and returns it at settlement, so its number tracks the request's whole lifetime. `tpm` reserves an estimated token cost and corrects it to the actual at settlement. The bottom row is the different one: at the gate it only reads, and writes nothing there, because its counter lives in the boxed region on the right, folded from ledger rows after the fact. A limit can be passed only if its counter is written after admission. That is the entire difference between the classes, and it predicts every other behaviour on this page. ## The four classes | class | limit types | counter written | can it be passed | across replicas | |---|---|---|---|---| | LedgerSettled | `budget_usd`, `quota_tokens`, `quota_requests` | off path, after the row lands | yes, by everything admitted since the last fold | converges when a shared Redis is configured; the fleet total lags one flush | | AdmissionCounted | `rpm` | at the gate, one way | no, within a replica | `local` ⇒ fleet ≤ N×limit; `redis` ⇒ fleet-exact while healthy; `share` ⇒ ÷N per replica | | ReserveReconcile | `tpm` | at the gate as an estimate, corrected at settlement | yes, when the estimate ran low | `redis` is turned away at boot; `local` per replica, or `share` for ÷N | | InflightLease | `concurrency` | at the gate as a lease, returned at settlement | no | replica-local gauge; `share` caps this replica at `ceil(C/N_live)` | ### LedgerSettled — spend and quota Counts money and volume: nanodollars for `budget_usd`, input plus output tokens for `quota_tokens`, forwarded rows for `quota_requests`, accumulated per scope over a calendar period. A worker off the request path, the accountant, folds each finished request's ledger row into those counters once the response is done. The gate itself writes nothing; it reads a published snapshot of the counter, republished on every ingest and on a five-second heartbeat, and trusted for thirty seconds before it counts as stale. So the number the gate reads is behind reality, and that lag is the overshoot. It covers every request in flight, every stream still producing tokens, and every row still queued on the bounded channel feeding the accountant or dropped when that 4,096-deep queue filled. Those last two are the `tee_lag_rows` and `dropped_rows` in the bound above. A burst is the worst case for this class: the whole burst is admitted against one snapshot. Per request the gate pays one atomic load and a map probe, with no I/O and no lock. `burst:` is accepted on these three types and read by nothing. ### AdmissionCounted — `rpm` Counts arrivals, one per request. The gate calls a probe of cost 1 against the algorithm cell and the write is the decision, so there is nothing to lag behind. Within one replica the cap holds: the cell is a compare-and-swap loop over a single integer. The one-way part matters. There is no refund on this class, so an arrival that was admitted stays counted even when the request goes on to be turned away by something else. Burst behaviour comes from the algorithm: `burst:` defaults to `amount`, so a drained `gcra` key admits a full burst at once and then one arrival per emission interval. With the default `store: local` the cost is a compare-and-swap against an in-memory cell and no I/O at all. `store: redis` makes it one round trip instead, bounded by a 5ms per-operation deadline that falls back to the local limiter and discloses `suspended` on the row. ### ReserveReconcile — `tpm` Counts tokens, but it has none to count at admission, so it counts an estimate. The gate reserves `input estimate + output component` against a `gcra` cell. A `tpm` limit is pinned to that algorithm, and it is also the one `rpm` runs by default. At settlement the limit commits the signed difference between the estimate and the usage that was recorded. Between those two writes the counter overstates by whatever was over-reserved, which is the safe direction: capacity is held that nobody used, and the correction gives it back. The unsafe direction is an under-estimate. A reconcile debit is committed unconditionally, with no admit-or-decline decision attached, so an under-estimated request pushes the window past its cap and the next caller meets a fuller window than the arithmetic suggested. Cost is the acquire at the gate plus one more write at settlement. `store: redis` is turned away when governance compiles at boot, because the distributed script is acquire-only and cannot carry the correction. ### InflightLease — `concurrency` Counts requests in flight right now: an integer gauge per scope key, spread over 256 shards. The gate takes a lease, settlement returns it, and the release is idempotent, so a double release cannot free two slots. The cap cannot be passed; the acquire loop declines at the cap. Being a live gauge is a trade. The number cannot lag, because there is no history in it, and for the same reason there is nothing to reconstruct: it is the present instant, and it drops to zero on restart. A burst meets the cap immediately, since there is no window to fill. Cost is one compare-and-swap at the gate and one decrement at the release. The gauge is per-replica; `store: share` divides the cap by the live replica count instead of sharing a counter. Do not read `observed` on a `concurrency_exceeded` body. It is hard-coded to `0` while the limit is genuinely full, so the sample above says `0` of `2` at the moment two requests were in flight. Read `limit` and the live gauge from `GET /tt/v1/me` instead. This is tracked in [what governance does not do](/docs/build/governance-gaps). ## A `tpm` reserve is a worst case, not a measurement Because a `tpm` limit has to decide before the tokens exist, its reserve is pessimistic by construction, and the arithmetic is worth knowing before you size one. The input half is the prompt bytes divided by four. The output half is `min(max_tokens, 16384)`, or the full 16,384 when the caller sent no `max_tokens` at all. The one-sentence request in the capture above carries no `max_tokens`, so it reserves 16,391 tokens: seven for its prompt estimate, 16,384 for an output nobody bounded. Against a 20,000-token window the success headers say so, and they ride every served response because `governance.keys.disclose_limits` defaults to on. ```console Ratelimit-Policy: "team_tpm";q=20000;w=60 X-Ratelimit-Limit-Tokens: 20000 X-Ratelimit-Remaining-Tokens: 3609 ``` The reserve is given back at settlement, so an over-reserve costs nothing beyond the seconds it was held. What it does cost is headroom, and the product guards the pathological case at boot rather than letting you discover it in production. `burst:` on a `tpm` limit must be at least the 16,384 reserve cap, or the process declines to start: ```text Error: governance: compile limits: govern: limit "team_tpm": tpm limit "team_tpm": burst 500 is below the reserve cap 16384 — an ordinary max-token reserve could never fit (a permanent 100% refusal); raise burst (default is amount 500) to at least the reserve cap ``` `burst:` defaults to `amount`, so this is also the practical floor on a `tpm` limit's amount. Note where that message comes from: `tokentriage validate` does not run any limit type's compile step, so it reports the config clean and the guard fires at boot. Callers that set `max_tokens` reserve less and meet fewer 429s; the other half of the reserve moves with the size of the prompt they send. ## Order decides which limit answers With several limits selected for a request, the engine walks them within a gate deepest node first, then by limit id, then by node id, and stops at the first one that is over. It compares no amounts. So when a loose deep limit and a tight shallow one are both over, the deep one is named on the 429. "The tightest limit wins" holds only in the weak sense that any breach stops the request. Class decides something coarser and earlier: which of the two gates a limit is evaluated at. AdmissionCounted and InflightLease run before the body is read, so they see the endpoint and nothing else. LedgerSettled and ReserveReconcile run after parsing, where the model, the provider and the token estimate exist. A request can spend an `rpm` arrival at the first gate and then be stopped by a budget at the second. Leases and reserves are handed back on that path; an `rpm` count is not, so your rate window drains slightly faster than your served traffic. To reconcile the two, read `refused_by.limit_id` off the decision ledger rather than inferring which limit fired from the 429 count on a node. ## The same split shows up in `govern preview` `govern preview` replays your recorded ledger against a proposed config, and it sorts its own output into three labels rather than presenting one confident list. Those three lines are below, cut out of a longer report with `# …` marking what was dropped; [turn governance on](/docs/build/governance-rollout#preview-the-cap-against-traffic-you-already-have) prints one in full, closing stanza included. ```console # … EXACT — LedgerSettled recomputation (hand-count-verifiable, exact): # … APPROXIMATE — rate-class (label: rate_windows_approximated): # … [concurrency] user_concurrency @ level:user CANNOT-ESTIMATE: concurrency requires overlapping-flight reconstruction; the ledger records no reliable per-request flight interval — cannot estimate from history # … ``` Three labels over four classes: LedgerSettled is EXACT, the two rate classes share APPROXIMATE, and InflightLease is CANNOT-ESTIMATE. The closing stanza asks the same question of the live counters, and it discloses its own lag while answering. Notice that this is the inverse of the live gate. At admission a spend cap is the approximate one and `rpm` is the precise one. Over history a spend recomputation is checkable by hand, because the ledger holds the rows it folds from, while window state cannot be rebuilt from row timestamps and per-request flight intervals were not recorded at all. Same four classes, same question about where the state lives, opposite answers. ## What this page does not cover Per-type config keys, defaults and validation errors are in [the six limit types](/docs/build/governance-limits), and the choice between `gcra`, `fixed_window`, `sliding_window` and `token_bucket` is in [rate algorithms](/docs/build/governance-rate-algorithms). Convergence bounds and what each store discloses belong to [governance across replicas](/docs/build/governance-distributed). Every field a 429 can carry is in [the refusal wire reference](/docs/build/governance-refusal-reference). Reading what was spent, and by whom, is reporting rather than enforcement: [cost analytics](/docs/build/cost-analytics) owns that.