The six limit types

Six limit types ship, and choosing between them is mostly a question about timing. Do you want a cap on what a team spends this month, or on how fast it can arrive this minute?

Those two answers are counted in different places. A monthly dollar cap is settled from your ledger after each request finishes, so it can drift past its number. A per-minute cap is counted at the door, before anything is forwarded, so it holds.

Each entry carries the config, the defaults, the cost, the failure mode, and the output that shows it working, in the order the product registers them.

If you have not met the four counting mechanisms, read how a limit counts first. This page names each type’s class and links there.

What each type caps

Type	Caps	Class	Unit	error.type	Status
budget_usd	dollars spent in a calendar period	LedgerSettled	usd	budget_exceeded	429
quota_tokens	input plus output tokens in a calendar period	LedgerSettled	tokens	quota_exhausted	429
quota_requests	forwarded requests in a calendar period	LedgerSettled	requests	quota_exhausted	429
rpm	requests admitted per window	AdmissionCounted	requests	rate_limited	429
tpm	tokens reserved per window	ReserveReconcile	tokens	rate_limited	429
concurrency	simultaneous in-flight requests	InflightLease	concurrent-requests	concurrency_exceeded	429

Every limit breach is a 429. One further 429 exists: quota_cost_unfittable, when a single request asks for more than the window’s burst can ever hold. Only tpm reaches it in this build, because an rpm probe costs one unit and a burst is at least one. Anything turned away for a reason other than a limit breach carries a different status, and the refusal wire reference has them all.

error.type names the family, and refused_by.limit_type names the type that fired, so rpm and tpm share rate_limited and stay distinguishable on the row.

Keys every type shares

These keys mean the same thing everywhere, so the per-type tables that follow list only what is specific to that type.

Key	Default	What it does
id	required	names the limit on every 429 and every row; charset [a-z0-9_]
attach.node or attach.level	required, one of the two	pins the limit to one node, or templates it at org, team or user
match.model, .provider, .endpoint	none	narrows which requests the limit counts
posture.mode	on	shadow records a would-refuse and admits; off is inert
posture.enforce	hard	soft admits on breach instead of returning a 429
posture.fail	open	closed with enforce: hard turns an uncheckable limit into a 503
posture.thresholds	none	alert fractions in [0,1]
override.allow	false	permits a per-node override, on three of the six types
override.max_amount, override.max_amount_usd	none	the ceiling an override may not pass; omitting it declares no ceiling at all

A limit with no posture: block is hard-enforcing and fail-open. Shadow is something you type, not something you get by leaving the block out.

Two of these keys have edges. match.model and match.provider read values the gate has not parsed yet on rpm and concurrency, so those two types can be narrowed only by match.endpoint. And attach.level also accepts global and key, neither of which instantiates a node in this build; a fleet-wide limit is written attach: { node: __global }. What governance does not do collects both, and the other keys that load cleanly and change nothing.

The six types

budget_usd

Caps the dollars a scope may spend inside a calendar period, folded from priced ledger rows.

- id: org_monthly_budget
  attach: { level: org }
  type: budget_usd
  amount_usd: 1000        # dollars, quantized to the nanodollar grid
  period: month           # day | week | month
  posture: { mode: "on", enforce: hard, fail: open }
  override: { allow: true, max_amount_usd: 5000 }

How it counts. A worker folds each settled row into a per-scope counter after the request has finished, and admission reads a snapshot of that counter. A priced row adds its cost. An UNPRICED row adds nothing to the dollar total and is still counted as a row. A cache replay adds a priced zero. That is the LedgerSettled class.

Knobs.

Key	Default	What it does
amount_usd	required	the cap in dollars; amount on this type is a config error
period	required	day, week or month; a rate window is a config error
algorithm, store	not accepted	either one is a config error on a spend or quota type
burst	not read	accepted by validation and consumed by nothing

What it costs. One snapshot read on the request path. Off the path, one counter per limit and scope key, fed from a bounded queue that drops and counts rows rather than slowing the proxy.

How it fails. It overshoots. The counter is written after the request that should have filled it, so spend already in flight is missing from the number admission reads. Here is the refused_by block of a captured 429:

{"limit_id":"org_monthly_budget","limit_type":"budget_usd","unit":"usd","node":"org:acme","period":"month","resets_at":"2026-09-01T00:00:00Z","retry_after_s":1912510,"observed":0.0000126,"limit":0.000001}

observed is 12.6 times the cap. That capture used a limit of one millionth of a dollar so a single request would pass it, which exaggerates the ratio; the mechanism is what generalises. When the accountant is stale or has not armed, the limit becomes uncheckable and the request is admitted with suspended:["org_monthly_budget"] on its row, unless the limit carries fail: closed with enforce: hard, which produces a 503 instead.

How to tell it is working. GET /tt/v1/me with the caller’s key returns each armed limit and its live numbers. Same sandbox, so the same millionth-of-a-dollar cap:

{"limit_id":"org_monthly_budget","type":"budget_usd","class":"spend","unit":"usd","node":"org:acme","period":"month","resets_at":"2026-09-01T00:00:00Z","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"limit_usd":{"usd":0.000001,"estimated":false,"unpriced":false},"observed_usd":{"usd":0.0000126,"estimated":true,"unpriced":false},"remaining_usd":{"usd":0,"estimated":true,"unpriced":false}}

Reach for something else when you need a cap that cannot be passed. A dollar figure is only known once the provider answers, so no spend limit can be a hard stop. rpm and concurrency are counted before the request leaves, and they hold.

quota_tokens

Caps the tokens a scope may consume inside a calendar period, counting input and output together.

- id: team_monthly_tokens
  attach: { level: team }
  type: quota_tokens
  amount: 50000000        # total tokens, input + output
  period: month
  posture: { mode: "on", enforce: hard, fail: open }

How it counts. The same off-path fold as budget_usd, metered in tokens. The whole token term is one line:

c.AddTokens(rec.Usage.InputTokens, rec.Usage.OutputTokens)

There is no metric key in this build, so input-only and output-only caps are not expressible. Two comments elsewhere in the source advertise metric: input|output|total; the type’s own header states total-only, and no such field exists on the config struct or the limit definition. LedgerSettled class.

Knobs.

Key	Default	What it does
amount	required	the cap as a token count; amount_usd on this type is a config error
period	required	day, week or month
algorithm, store, burst	not accepted or not read	two config errors and one key nothing reads

What it costs. Nothing on the request path. Each limit keeps its own counter per scope key, and each settled row is folded once per selected instance.

How it fails. It lags the same way budget_usd does. Two details differ. An UNPRICED row still consumes tokens, because only budget_usd excludes unpriced work from its total. A cache replay consumes zero tokens and still counts as a row.

quota_tokens sums the row’s input and output tokens; tpm settles against the provider-reported total, a separate field that can include reasoning tokens. The two can disagree about the same request.

How to tell it is working. The EXACT block of govern preview recomputes it from ledger history. Its line for this limit, over the five-row sandbox ledger used later on this page:

  [quota_tokens] team_monthly_tokens @ org:acme/team:ml  observed 60 / ceiling 50000000  (frac 0.0000, rows 1)

Reach for something else when you care about money rather than volume. Token counts across models are not comparable spend, and budget_usd prices each row through the catalog.

quota_requests

Caps forwarded requests in a calendar period. Its entire fold is one row increment.

- id: team_daily_requests
  attach: { level: team }
  type: quota_requests
  amount: 20000           # forwarded requests
  period: day
  posture: { mode: "on", enforce: hard, fail: open }

How it counts. Every forwarded row counts once, whether priced, UNPRICED, or served from cache:

func (l *compiledQuotaRequests) Fold(rec *declog.Record, c *govern.Counters) {
	c.AddRow(rec.Cost.Unpriced, rec.Cost.Estimated)
}

A row for a request that was turned away is skipped before the fold, so a storm of 429s consumes no quota. LedgerSettled class.

Knobs.

Key	Default	What it does
amount	required	the cap as a request count
period	required	day, week or month
algorithm, store, burst	not accepted or not read	two config errors and one key nothing reads

What it costs. Nothing on the request path, and one integer per scope key off it.

How it fails. It lags, and a cache hit consumes a slot. A caller served entirely from your cache spends no money and still spends quota.

How to tell it is working. The same EXACT block:

  [quota_requests] team_daily_requests @ org:acme/team:ml  observed 1 / ceiling 20000  (frac 0.0001, rows 1)

Reach for something else when you want to pace arrivals rather than cap a period’s volume. That is rpm.

rpm

Caps requests admitted per rolling window, counted at the gate before the body is read. rps is this type with period: 1s, not a separate registration.

- id: team_rpm
  attach: { level: team }
  type: rpm
  amount: 300             # requests per window
  period: 1m              # a window duration, not a calendar period
  posture: { mode: shadow }

How it counts. One algorithm step per request through a counter store, taken at the first gate. The number is the admission decision, so it cannot lag. The step is one-way: a probe that admitted stays consumed even when a later limit stops the same request. AdmissionCounted class.

Knobs.

Key	Default	What it does
amount	required	requests admitted per window
period	required	any Go duration, such as 1s or 1m; a calendar period is a config error
burst	amount	how many arrivals a drained key absorbs at once
algorithm	gcra	also fixed_window, sliding_window, token_bucket
store	local	also redis, which needs shared_state.redis, and share

Burst defaults to the full amount, so a drained key admits amount arrivals at once and then one per emission interval. Rate algorithms covers what each choice guarantees, the window algorithms’ 65,535 ceiling, and the gcra requirement on store: share.

What it costs. With the default local store: a read lock on one of 256 shards to find the key’s cell, then a lock-free compare-and-swap loop over a single int64. Memory is one cell per scope key, reclaimed by a collector that runs only when a shard gains a new key. With store: redis, one scripted round trip on the request path under a 5 ms deadline.

How it fails. Two ways. A limit with no period: passes tokentriage validate and then stops the boot:

Error: governance: compile limits: govern: limit "team_rpm": rpm requires a rate-class window period (e.g. 1s, 1m), not a calendar period

Both governance example configs in the product repo have that shape.

Across replicas with the default store, each replica keeps its own cell, so a fleet of N admits up to N times the amount. That bound is published rather than hidden; governance across replicas has the arithmetic.

How to tell it is working. With governance.keys.disclose_limits at its default of true, a served response advertises the tightest rate limit:

HTTP/1.1 200 OK
Content-Type: application/json
Ratelimit-Policy: "team_rpm";q=300;w=60
X-Ratelimit-Limit-Requests: 300
X-Ratelimit-Remaining-Requests: 59
X-Tt-Request-Id: tt_tbuhetygwddxiaprkw3j3fkwze

That capture came from a limit configured with burst: 60, and Remaining tracks the burst depth rather than the 300 per minute. The limit was in mode: shadow and still advertised its policy.

Reach for something else when request cost varies by orders of magnitude. Three hundred one-token requests and three hundred 100,000-token requests look identical to rpm. tpm prices the window in tokens.

tpm

Caps tokens per rolling window by reserving an estimate at the gate and settling it to observed usage afterwards.

- id: team_tpm
  attach: { level: team }
  type: tpm
  amount: 200000          # tokens per window
  period: 1m
  match: { model: "gpt-*" }
  posture: { mode: "on", enforce: hard, fail: open }

How it counts. After the body is parsed, the gate reserves est_input + min(max_tokens, 16384) tokens, where est_input is the prompt byte count divided by four. With no max_tokens on the request, the reserve is est_input + 16384. At settlement the difference between the reserve and the observed total is committed as a signed adjustment, with no second admission check. ReserveReconcile class.

Knobs.

Key	Default	What it does
amount	required	tokens per window
period	required	any Go duration; a calendar period is a config error
burst	amount	must be at least 16384, the reserve cap
algorithm	gcra	the only value that compiles
store	local	also share; redis is turned down at compile

What it costs. One write to the cell at the gate and, unless the estimate matched observed usage, a second at settlement, plus one reservation held for the life of the request. Every tpm 429 is flagged approximate: true, because a reserve is an estimate.

How it fails. Three ways. A burst under 16,384 does not compile, and burst defaults to amount, so a limit smaller than the reserve cap cannot be built at all:

Error: governance: compile limits: govern: limit "team_tpm": tpm limit "team_tpm": burst 500 is below the reserve cap 16384 — an ordinary max-token reserve could never fit (a permanent 100% refusal); raise burst (default is amount 500) to at least the reserve cap

The guard reads the effective burst, so type: tpm with amount: 10000 and no burst: stops the same way. Raise amount to 16,384 or more, or set burst: 16384 and keep the smaller rate.

Second, a request whose reserve exceeds the burst can fit at no future moment, and it gets quota_cost_unfittable with no useful retry hint.

Third, settlement commits a debit past the cap. An under-estimate is corrected after the fact, so the window can go over by the size of the miss.

How to tell it is working. A served response advertises the token pair:

Ratelimit-Policy: "team_tpm";q=20000;w=60
X-Ratelimit-Limit-Tokens: 20000
X-Ratelimit-Remaining-Tokens: 3609

That capture came from a limit configured with amount: 20000, and 20000 − 16391 = 3609 is one request holding a 16,391-token reserve.

Reach for something else when you want a calendar-period token cap rather than a per-minute one; that is quota_tokens. Also when a fleet total has to be exact, since tpm cannot bind store: redis and an rpm limit can.

concurrency

Caps how many requests a scope may have in flight at the same instant.

- id: user_concurrency
  attach: { level: user }
  type: concurrency
  amount: 8               # simultaneous in-flight requests
  posture: { mode: "on", enforce: hard, fail: open }

How it counts. A live integer per scope key, incremented at the first gate and decremented once when the request finishes. No window, no algorithm, no counter store. The release is idempotent, so a double release cannot free two slots, and settlement runs on every terminal path including a client disconnect. InflightLease class.

Knobs.

Key	Default	What it does
amount	required	maximum simultaneous in-flight requests per scope key
store	local	also share, which caps this replica at ceil(amount / N_live)
algorithm	not accepted	a config error on this type
burst	not read	accepted by validation and consumed by nothing
period	not read	a window value loads and changes nothing at the gate; a calendar value is a config error

A window period: here is worse than inert. The period is one of the components a counter’s scope key is built from, so adding or removing a period: that no code reads still splits the live gauge into two.

What it costs. A compare-and-swap on one int64 gauge at the gate and one atomic decrement at settlement. Memory is one gauge per scope key, held for the life of the process rather than reclaimed.

How it fails. The gauge is per replica and lives in memory, so a restart drops it to zero and a rolling restart briefly admits more than the cap. And the 429 body reports observed: 0 beside a live limit. This capture is from a sandbox limit at amount: 2 with four requests fired at once against a 3-second upstream:

{"error":{"type":"concurrency_exceeded","message":"request refused by limit user_concurrency","refused_by":{"limit_id":"user_concurrency","limit_type":"concurrency","unit":"concurrent-requests","node":"org:acme/team:ml/user:dev1","observed":0,"limit":2},"enforcement":{"posture":"enforcing","approximate":false}}}

Two requests were in flight at that moment, so treat observed on this type as unpopulated and read the gauge from /tt/v1/me instead. There is also no Retry-After worth acting on, because a slot frees when a request finishes rather than on a clock.

How to tell it is working. The /tt/v1/me row carries the live gauge and the cap this replica is enforcing:

{"limit_id":"user_concurrency","type":"concurrency","class":"count","unit":"concurrent-requests","node":"org:acme/team:ml/user:dev1","period":"","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"amount":10,"observed":0,"remaining":10}

That capture came from a limit configured with amount: 10 and no traffic in flight.

Reach for something else when the problem is arrival rate rather than simultaneity. One slow client holding two connections and one fast client making 600 calls a minute are different problems, and rpm addresses the second.

Compiling the set before it boots

Most of the guards named in this page fire at compile, and compile happens at boot. You want that answer before a deploy, not during one.

Preview also recomputes each limit over your ledger, which is the fastest way to see all six armed at once. Every armed limit gets a line of its own under one of three labels, and the run below is cut back to one label and one of its lines, with # … marking what was dropped; turn governance on prints a report in full.

$ tokentriage govern preview -c sixlimits.yaml --log ./gov/state2/decisions.jsonl
# …
EXACT — LedgerSettled recomputation (hand-count-verifiable, exact):
  [quota_requests] team_daily_requests @ org:acme/team:ml  observed 1 / ceiling 20000  (frac 0.0001, rows 1)
# …

That ran against the six configs on this page and a five-row sandbox ledger, so read the labels rather than the figures.

Config errors this preview catches that validate misses

Each of these loads clean under tokentriage validate and stops the compile:

• rpm or tpm with no period: at all
• tpm with algorithm: sliding_window, or any value other than gcra
• tpm with an effective burst under 16,384
• tpm with store: redis alongside a shared_state.redis block, which reports type "tpm" (reserve/reconcile) cannot bind store "redis"
• rpm with algorithm: fixed_window or sliding_window and an amount above 65,535

One class of error is caught by neither command. Preview resolves a store name without building the store, so a bad algorithm-to-store pairing — store: share on any algorithm other than gcra, or token_bucket naming share or redis — surfaces only at boot. The tpm and redis row above is different: that one is a store-class rule the compiler checks, which is why preview reports it.

validate does catch six shapes on its own: store: on a spend or quota type, algorithm: on a spend or quota type, algorithm: on concurrency, store: redis on concurrency, a calendar period: on a rate type, and store: redis on rpm or tpm with no shared_state.redis block.

Per-node overrides reach three of the six

A template gives every node at its level the same cap. When one node needs a different number, whether you can give it depends on the type. A per-node override may raise or lower budget_usd, quota_tokens and quota_requests. An override on rpm, tpm or concurrency is turned down, because those three bake the amount into a window store or a gauge when they compile and do not read a per-node value.

GET /tt/api/v1/governance/limits states that rule as data: every armed definition carries override_allowed. This response is from a sandbox config with a tiny shadow rpm, not from the samples above, so read the flags rather than the amounts:

{
  "definitions": [
    {"limit_id":"org_monthly_budget","limit_type":"budget_usd","class":"spend","unit":"usd","period":"month","posture":"enforcing","attach":"template","level":"org","amount":{"usd":1000,"estimated":false,"unpriced":false},"override_allowed":true,"override_max":{"usd":5000,"estimated":false,"unpriced":false}},
    {"limit_id":"team_rpm","limit_type":"rpm","class":"count","unit":"requests","period":"1m0s","posture":"alert_only","attach":"template","level":"team","amount_count":2,"override_allowed":false},
    {"limit_id":"team_tpm","limit_type":"tpm","class":"count","unit":"tokens","period":"1m0s","posture":"enforcing","attach":"template","level":"team","amount_count":20000,"override_allowed":false},
    {"limit_id":"user_concurrency","limit_type":"concurrency","class":"count","unit":"concurrent-requests","posture":"enforcing","attach":"template","level":"user","amount_count":10,"override_allowed":false}
  ],
  "overrides": [],
  "note": "definitions are the ARMED limits enforced now; overrides are persisted runtime writes that take effect on the NEXT governance reload (compile-bound). No override is reported as live-enforced here."
}

An override attempted on one of the other three is dropped with a reason naming the type:

limit type "rpm" does not support per-node overrides (its cap is enforced by a per-window/gauge store that cannot honor a per-node amount); overrides apply only to spend/quota limits (budget_usd, quota_tokens, quota_requests)

override.max_amount and override.max_amount_usd declare a ceiling for the template rather than a per-node instance, and omitting the ceiling while setting allow: true leaves any positive amount acceptable. The tenancy hierarchy owns the rest of the mechanism.

What this page does not cover

The types, not the machinery around them. What each class implies under burst and across replicas is how a limit counts. What each algorithm guarantees is rate algorithms. Which instances a request selects is the tenancy hierarchy. Every field on a 429 is the refusal wire reference. What was spent, by whom, and how it was attributed is cost analytics.