Why was this request refused?

Someone pastes a 429 into your issue tracker. It has a status line, a handful of headers and a JSON body, and what they want to know is which cap they hit, whose cap it was, and whether you can raise it. All three answers are already in front of you.

This page works outward from that response. First the headers that name the reason and the limit. Then the body field that names the tenant the cap belongs to. Then the ledger row, which carries the same block under the same request id. After that come the traps, and a table from what you were handed to the config key that changes it.

It assumes governance is armed on your gateway and that you know roughly what a limit is; Governance is the front door, and the six limit types covers what each cap counts. The complete field contract is every field a governance stop can carry; this page is the diagnosis rather than the catalogue.

Read the headers first

The headers are the cheapest answer available, and one of them rides every governance stop whether or not a limit was involved.

HTTP/1.1 429 Too Many Requests
Content-Type: application/json
Ratelimit: "team_rpm";r=0;t=20
Retry-After: 20
X-Tt-Refusal-Type: rate_limited
X-Tt-Refused-By: team_rpm
X-Tt-Request-Id: tt_e6gbpncee32dbu7y5jpsfr4ax4

{"error":{"type":"rate_limited","message":"request refused by limit team_rpm","refused_by":{"limit_id":"team_rpm","limit_type":"rpm","unit":"requests","node":"org:acme/team:ml","period":"1m0s","retry_after_s":20},"enforcement":{"posture":"enforcing","approximate":false}}}

That capture is a source build driving a local sandbox with a stand-in upstream, so the request id and the limit ids are that machine’s. The header names and the body shape are what your own callers get.

header	what it tells you	when it is present
x-tt-refusal-type	the reason, from a closed set of eight	every governance stop, without exception
x-tt-refused-by	the id of the limit that fired	only where one named limit fired, so absent on a 401, a 403, and a 503 that names none
x-tt-request-id	the id of this request’s ledger row	every response, stopped or served
retry-after	whole seconds until a retry could succeed	a 429 whose limit knows a time; a 503 carries none
ratelimit	the IETF structured field, "<limit id>";r=0;t=<reset seconds>	a 429 naming a limit

Read them in that order. x-tt-refusal-type says what kind of thing stopped the request, x-tt-refused-by says which one, and x-tt-request-id gets you everything else. Header names are case-insensitive and the server canonicalises them on the way out, which is why the capture shows X-Tt-Refusal-Type where the table shows x-tt-refusal-type. Match them case-insensitively.

Then the body, which names the tenant

The header names the limit. The body names whose limit it is, and that is usually the question behind the question.

error.refused_by is one object, and node is the field most people are looking for without knowing it. On a breached cap it carries the tenancy node that cap is attached to, not the caller’s own node. In the preceding capture the sandbox’s key sits at org:acme/team:ml/user:dev1 and the cap that stopped it sits at org:acme/team:ml, so this is the team’s rate cap. Raising that user’s own limit would change nothing.

field	what it answers
limit_id	which definition fired, matching x-tt-refused-by
limit_type	which of the six types it is, so you know what it counts
unit	usd, tokens, requests, or concurrent-requests
node	the tenancy node the cap is attached to
period	a calendar period like month, or a window like 1m0s
resets_at	the calendar boundary, on a calendar period only
retry_after_s	seconds, matching the Retry-After header
observed and limit	the breach numbers, present together or not at all

Two of those rows are conditional, and the conditions trip people up. resets_at appears on a calendar period and is absent on a rolling window. observed and limit are emitted as a pair, and a rate stop omits both, rpm and tpm alike, because their counter store cannot report the pair cheaply. Their absence is not a defect. The sibling enforcement block is the class disclosure, and how a limit counts reads it out.

The whole refused_by object is dropped on a 401, and on a 503 that names no limit, so node is guaranteed only where refused_by itself is. On a 403 it holds the caller’s own node, since no cap was involved.

The row carries the same block

Everything so far is also on disk, under the same request id, and it got there from the same value in memory.

One function stamps the response headers, the response body, and the decision row’s refused_by_limit block before any byte goes out. A row and the response the caller saw cannot disagree, and nothing is stopped on the wire without a row being written. Take the x-tt-request-id off the paste and search your decision log for it.

Here is the row a 401 wrote when a presented virtual key did not resolve:

{
  "http": { "path": "/v1/chat/completions", "dialect": "openai", "status": 401, "endpoint": "chat" },
  "decision": {
    "governance": { "kid": "", "node": "", "verdict": "allow", "gate_us": { "a": 0, "b": 0 } },
    "refused_by_limit": { "type": "invalid_key" }
  }
}

Three fields mark a request that did not go through. Two of them are on that excerpt: http.status, and the presence of decision.refused_by_limit. The third is outcome.finish_reason, which reads limit_refused on every governance stop.

The verdict field does not say refused

Look at the row again. The caller got a 401, and decision.governance.verdict reads allow. That is not a bug in the capture, and it will cost you an afternoon if you meet it in a query instead of here.

verdict holds two values and only two, allow and would_refuse. There is no third value for a stopped request. The field reports what shadow evaluation found: would_refuse names limits in mode: shadow that were over their cap and were recorded rather than acted on. A shadow breach stops nothing, so a row carrying would_refuse was admitted by governance; where such a row is also a stop, a different enforcing limit did the stopping. Governance shows a shadow row in full.

Symptom, cause, and what to change

Eight reasons ship, and the status follows from the reason rather than the other way round. Find your x-tt-refusal-type in the first column.

error.type	status	what happened	what to change
invalid_key	401	the presented ttv_ did not resolve, or none was sent while require_key is required	find the kid with tokentriage keys list and read STATE; a lapsed key still lists as active, so read EXPIRES too
scope_violation	403	the endpoint, or the served model or provider, is outside the key’s scope	re-mint with the value added to --models, --providers or --endpoints; scopes cannot be edited in place
rate_limited	429	an rpm or tpm window is full, and limit_type says which	raise amount: on the named limit, raise burst: for spiky traffic (only gcra and token_bucket read it), or wait retry_after_s
quota_exhausted	429	a quota_tokens or quota_requests cap is spent for the period	raise amount:, or wait for resets_at; nothing is carried forward across a period boundary
budget_exceeded	429	a budget_usd cap is spent for the period	raise amount_usd:, or store a per-node override where the template declares override: { allow: true, max_amount_usd: … }, which enforces at the next governance reload
concurrency_exceeded	429	the node is at its in-flight amount	raise amount:; a lease frees when a request finishes, so no clock helps
quota_cost_unfittable	429	one request’s token reserve is larger than the whole tpm burst, so no wait admits it	raise burst: to at least the reserve the caller takes, or have the caller send a smaller max_tokens
enforcement_unavailable	503	the enforcement question could not be answered, and something was set to stop rather than serve	clear the underlying fault, or set posture: { fail: open } on the limit named in x-tt-refused-by

Two rows have narrower causes than they look. quota_cost_unfittable can only come from a tpm limit: the one other rate type, rpm, probes at a cost of one unit, and one unit always fits a burst floored at one. And scope_violation names the violated dimension in error.message, with a refused_by carrying only the holder’s own node, so the body will not list what the key is allowed to reach.

Which gate stopped it, and in what order

A limit’s cap is not the only thing that decides the answer. Where the limit is evaluated decides what the caller can be told at all, and which of several over-cap limits gets named.

Read the left column downward as one request and the right column as what the caller can be told at that point. StageA runs before a byte of the body is parsed, so it holds the credential and the endpoint but not the model: it is the only moment that can answer 401, and it is where rpm and concurrency caps are evaluated. StageB runs after parsing and routing, where spend and quota are probed and the tpm reserve is taken. ServedCheck re-asserts model and provider scope against the model that will serve. Settlement stops nothing, because the outcome is already decided.

Inside one gate, the engine orders the selected instances by node depth descending, then limit id ascending, then node id ascending, and stops at the first one over its cap. It compares no amounts. The sandbox behind the captures on this page resolved four caps for one key, which is enough to see both consequences:

1. user_concurrency at org:acme/team:ml/user:dev1, depth 3, evaluated at StageA
2. team_rpm at org:acme/team:ml, depth 2, evaluated at StageA
3. team_tpm at org:acme/team:ml, depth 2, evaluated at StageB
4. org_monthly_budget at org:acme, depth 1, evaluated at StageB

The first consequence is that the deepest limit that is over gets named, not the tightest. team_tpm and org_monthly_budget are both evaluated at StageB, so when both are over, the 429 says team_tpm, because it sits a level deeper. The org budget can be the harder wall and still go unmentioned. All four of those caps produced a 429 of their own against this sandbox as their counters moved, each naming itself.

The second consequence is quieter. team_rpm counts the arrival at StageA and there is no refund on that class, so a request that goes on to be stopped by org_monthly_budget at StageB has already spent its arrival. Leases and reserves are handed back at settlement; a counted arrival is not. Your rate window therefore drains slightly faster than your served traffic, which is expected rather than drift.

Two answers that are not a breached cap

A 401 tells the caller almost nothing, and it withholds on both surfaces. There is no limit id, no node, and no refused_by object, on the wire or on the row. One uniform message covers an unknown key, a revoked key and an expired one alike. So a caller cannot use a 401 to discover whether a tenant or a credential exists, and you cannot diagnose one from the response body either. Go to tokentriage keys list and the key’s STATE and EXPIRES columns instead.

A 503 enforcement_unavailable is not a breached cap. It means the gate could not answer the enforcement question and was configured to stop rather than serve. Two paths reach it: a limit that could not be checked while carrying fail: closed with enforce: hard, and require_key: required on a replica with no loadable key snapshot, which a cold replica can serve for the moments before its key store seeds. The first names its limit in x-tt-refused-by; the second names nothing, because no limit was involved. Neither carries a Retry-After, since there is no time at which the retry is known to work. Every other internal fault degrades to an allow with a disclosure, which Governance covers.

When the numbers look wrong

A budget_usd stop can report an observed larger than its own limit, and that is the design rather than a fault:

{"error":{"type":"budget_exceeded","message":"request refused by limit org_monthly_budget","refused_by":{"limit_id":"org_monthly_budget","limit_type":"budget_usd","unit":"usd","node":"org:acme","period":"month","resets_at":"2026-09-01T00:00:00Z","retry_after_s":1912510,"observed":0.0000126,"limit":0.000001},"enforcement":{"posture":"enforcing","approximate":true,"bound":{"tee_lag_rows":0,"dropped_rows":0},"note":"spend from in-flight and streaming requests admitted before this refusal may not yet be reflected"}}}

observed is 0.0000126 against a limit of 0.000001, so the counter was 12.6 times past its cap when it acted. The dollar figures are arithmetic over a stub upstream’s token counts and are nobody’s bill. The body says why in the same breath: approximate: true, plus a note naming what is missing from the number. A spend counter is folded from ledger rows off the request path, so admission reads a snapshot that does not yet contain anything still in flight. How a limit counts has the mechanism and the size of the gap.

Reading refusals in bulk

One 429 answers one question. A pattern needs the feed, and two admin-plane endpoints serve it. Both take a node query parameter that scopes the answer to a subtree, and both sit at the viewer role floor.

GET /tt/api/v1/governance/refusals is the live view: newest first, limit defaulting to 100 and clamped at 1000, cursor-paginated, with the older offset parameter deprecated because an offset into a growing feed re-reads and skips rows. It is bounded at 1000 rows and page.total saturates there, so treat it as the recent window rather than the archive. Each item carries ts, req_id, node, kid, type, limit_id, limit_type, posture, approximate and http_status, and a 401 item carries no limit_id for the same reason the wire does not.

GET /tt/api/v1/governance/exports/refusals is the archive twin: complete by default, window-scoped, and available as JSON or CSV through format=. The CSV columns are frozen in the order of that field list, and the file opens with #-prefixed comment lines that are the export’s provenance stamp; strip the # and the block is the JSON header object, carrying the schema version, generated_at, the instance, the window, the row count and a demo flag. A partial scan appends a # truncated: true line rather than presenting a short file as whole. The management API has both operations in full.

What this page does not cover

The field-by-field contract, including every header and every conditional field, is the refusal wire reference. Why a counter can be passed before it acts is how a limit counts. Which caps a given key resolves to in the first place, and how an override raises one, is the tenancy hierarchy. Turning a limit on without stopping traffic while you learn its shape is turn governance on.

What a tenant spent, as opposed to what stopped them spending it, is reporting rather than enforcement: cost analytics owns that.