The tenancy hierarchy

Three caps should hold on the same request at once: a monthly ceiling for the whole company, a tighter daily one for the team running the nightly batch, and a rate cap on one laptop so a runaway loop can’t eat either. When one of the three stops that request, you want to know which one, and why the other two stayed quiet.

All three hold because each request is resolved against a tree of tenancy nodes, and every cap the tree selects binds it. This page is the shape of that tree, the walk that turns it into a set of caps, and the one mechanism for giving a single node a bigger number than its neighbours.

It assumes you know what the six limit types count; The six limit types is that page. What follows is about which caps apply to a request, not what each one measures.

One example runs throughout: an org called acme with teams ml and search, two users under ml, and a virtual key held by dev1.

The tree, and where it comes from

A cap has to name who it applies to, and “a customer” is rarely one thing. It is a company, the teams inside it, the people inside those, and the credentials they hold. Governance models that as four levels (org, team, user, key), and you declare the first three.

governance:
  enabled: true # default false; an absent block is off
  tenants: # the declarative plane
    - org: acme
      teams: [ml, search]
      users: { ml: [dev1, dev2] }

That block materialises five nodes: org:acme, org:acme/team:ml, org:acme/team:search, org:acme/team:ml/user:dev1 and org:acme/team:ml/user:dev2. Every team you name under users: has to be declared in teams: as well; a users: group naming a team that is not stops tokentriage validate with governance.tenants[0].users["ml"]: names a team not declared in teams.

A node id is one or more <kind>:<id> segments joined by /. The kind is lowercase letters; the id is lowercase letters, digits, - and _. Uppercase, spaces and a .. segment are out of charset. That charset is checked where a delegated admin is authorised against a subtree, before the two ids are compared, so org:acme/../org:evil cannot be walked into another org’s tree. It is not checked where a node or a key is created: keys mint --owner and the management API’s node create both take the string as given, so a typo in either lands an identity whose chain resolves against nodes that do not exist.

__global is the synthetic root. You cannot declare it in tenants: and it is not a tree node, but every chain ends at it, and a limit pinned there applies to everything the gateway serves, including traffic carrying no key.

Nodes arrive from two places. governance.tenants is the declarative plane, edited in your config file and hot-reloadable. Nodes created through the management API are the runtime plane, stored in governance.db. At each arm both fold into one tree, so a runtime-created node can carry a config-declared template. When the same node id appears in both, the config spec is kept and the runtime row is skipped: no error, no warning. Creating a runtime node also has no effect on the live gate until the next governance reload.

How a limit attaches

You do not write a limit per tenant. A limit attaches once, in one of two forms, and the tree does the fan-out.

• Pinned — attach: { node: org:acme/team:ml } gives one instance, at that node.
• A level template — attach: { level: team } gives one instance per tree node at that level. Add a team to tenants: and it gets its own counter, with no edit to the limit.

Config requires one of node or level, and only one; setting both or neither stops tokentriage validate with governance.limits[0].attach: must set exactly one of {node, level}. A pinned node has to be in the tree when governance compiles, and that one is not a validate-time check. validate accepts a pinned node that does not exist; boot is where it stops, with governance: compile limits: govern: limit "ml_daily": pinned node "org:acme/team:nope" not in tenancy tree. __global is the exception, permitted as a pinned node without tree membership, and that is the working spelling for a fleet-wide cap.

governance:
  limits:
    - id: acme_month # a template: one instance per org
      attach: { level: org }
      type: budget_usd
      amount_usd: 1000
      period: month
      override: { allow: true, max_amount_usd: 5000 }
    - id: ml_daily # pinned to this node and no other
      attach: { node: "org:acme/team:ml" }
      type: budget_usd
      amount_usd: 200
      period: day
    - id: user_rpm # a template: one instance per user
      attach: { level: user }
      type: rpm
      amount: 120
      period: 1m
    - id: global_rpm # the spelling that reaches every request
      attach: { node: "__global" }
      type: rpm
      amount: 5000
      period: 1m

That set passes tokentriage validate and compiles under tokentriage govern preview. None of the four declares a posture: block, and an omitted one means hard enforcement rather than off; Turn governance on covers the ladder.

Read the tree down the left and the limits down the right; the dashed edges are limits binding to a level. The key at the bottom is drawn as data rather than as a tree node, and that is accurate. A key is counted through the node it belongs to.

How one request resolves

A key can print its own chain, and that chain is the resolution walk. GET /tt/v1/me answers with it:

$ curl -H "Authorization: Bearer $TT_KEY" http://127.0.0.1:8790/tt/v1/me

{
  "kid": "vk_FEKZdMs2UQRi",
  "node": "org:acme/team:ml/user:dev1",
  "chain": ["org:acme/team:ml/user:dev1/key:vk_FEKZdMs2UQRi","org:acme/team:ml/user:dev1","org:acme/team:ml","org:acme","__global"],
  "scopes": {"models":null,"providers":null,"endpoints":null,"allow_passthrough":false},
  "limits": [
    {"limit_id":"org_monthly_budget","type":"budget_usd","class":"spend","unit":"usd","node":"org:acme","period":"month","resets_at":"2026-09-01T00:00:00Z","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"limit_usd":{"usd":0.000001,"estimated":false,"unpriced":false},"observed_usd":{"usd":0.0000126,"estimated":true,"unpriced":false},"remaining_usd":{"usd":0,"estimated":true,"unpriced":false}},
    {"limit_id":"user_concurrency","type":"concurrency","class":"count","unit":"concurrent-requests","node":"org:acme/team:ml/user:dev1","period":"","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"amount":10,"observed":0,"remaining":10},
    {"limit_id":"team_rpm","type":"rpm","class":"count","unit":"requests","node":"org:acme/team:ml","period":"1m0s","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"amount":3,"observed":2,"remaining":1},
    {"limit_id":"team_tpm","type":"tpm","class":"count","unit":"tokens","node":"org:acme/team:ml","period":"1m0s","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"amount":20000,"observed":0,"remaining":20000}
  ]
}

chain carries five entries, deepest first. The first is the key leaf, formed as <owner node>/key:<kid>. Then the three cumulative prefixes of the owner node. Then __global, where every chain terminates. The limits array is the caller’s resolved set with each cap’s observed and remaining figures; those four ids come from the sandbox the capture was taken on, not from the config above.

Resolution turns that chain into a set of instances. Each step does one thing to the answer, and steps 1 to 6 read compiled metadata only, not a live counter.

1. Walk the limits in ascending id. The compiled order is lexical, so the order you wrote them in your config does not change the outcome.
2. Drop a limit whose match: the request does not satisfy. The test runs once per limit, before any node fan-out, and a dropped limit produces no instance and shows up in no probe trace.
3. Expand each surviving limit into nodes. A pinned limit yields its one node; a template yields every tree node at its level, in lexical order.
4. Keep only the nodes in this request’s chain, together with __global. A node outside the chain is dropped here, so a limit sitting on another tenant’s node contributes no instance to this request.
5. Substitute the per-node override, when one is stored for that limit and that node. It replaces the amount and nothing else.
6. Order the surviving instances by node depth descending, then limit id ascending, then node id ascending.
7. Evaluate them in that order, and stop at the first instance that is over its cap.

Step 7 is a conjunction. Every selected instance binds this request, so an org cap binds every team and user beneath it. It binds them because the org’s own instance is in the set, not because anything compares it against the team’s.

The diagram puts four caps on that same chain and works down it, with acme_month lowered to $50 so the ordering is visible. The key leaf selects nothing. user_rpm at depth 3 is under its cap and admits. ml_daily, the team’s $200/day budget, is over, so the request gets a 429 naming ml_daily. The org’s $50 monthly cap is over too, and it is the tighter number, but it sits shallower and is not reached on this pass.

The drawing flattens one thing. A rate cap is tested before the request body is read and a budget after it, so user_rpm and global_rpm are settled at the first gate and the two budgets at the second. Within a gate the order is the one drawn.

What the chain looks like for traffic with no key

An anonymous request carries a chain of one entry, __global. Every template is therefore out of chain, and so is every limit pinned at a tenant node. The only caps that apply are the ones pinned at __global whose match: the request satisfies. Worth knowing before you leave require_key at off and expect an org template to hold the line.

The tightest cap is not the one that names the 429

The operator guide that ships in the product repository, docs/GOVERNANCE.md, says resolution walks the chain as a conjunction and the tightest applicable limit wins. The first half is right. The second is not what the engine does.

Amounts are not compared at any point. Instances are ordered by depth, then limit id, then node id, and evaluation stops at the first breach. So when a loose deep cap and a tight shallow cap are both over, the deep one is what lands in refused_by and in the X-Tt-Refused-By header.

Tightest-wins holds in one weak sense: any breach stops the request, so the effective ceiling is the smallest applicable cap. What it misleads about is which limit gets named. That is the deepest one that was over, which can be the loosest. Read more than the one 429: GET /tt/v1/me lists every cap the caller resolves, so a shallower cap sitting at zero is visible even while a deeper one does the talking.

Raising one node’s cap

One team needs a bigger budget than the template gives it, and forking the limit into a second definition leaves two things to keep in step. An override is the alternative: it changes the amount of one instance and leaves every other node on the template.

The template declares the permission and the ceiling. allow defaults to false and max_amount defaults to 0:

override: { allow: true, max_amount_usd: 5000 }

The config plane carries only that pair. The per-node override itself is a runtime-plane row, written through the management API and audited.

Three more things are worth knowing before you rely on one.

An override moves spend and quota only. It is honoured by budget_usd, quota_tokens and quota_requests. On rpm, tpm and concurrency it is turned down with its reason, because those bake their amount into a per-window store or a live gauge when the limit compiles and do not read the instance’s amount afterwards. The management API states the rule as data:

{"limit_id":"team_rpm","limit_type":"rpm","class":"count","unit":"requests","period":"1m0s","posture":"alert_only","attach":"template","level":"team","amount_count":2,"override_allowed":false}

To vary a rate cap per node, pin a separate limit at that node.

A runtime override does not enforce until the next governance reload. The write is persisted and audited straight away, and it is disclosed as pending. The API says so itself:

definitions are the ARMED limits enforced now; overrides are persisted runtime writes that take effect on the NEXT governance reload (compile-bound). No override is reported as live-enforced here.

The armed definition keeps showing the amount in force, so lowering a cap through the API cannot read as applied while traffic still flows against the old number.

One bad override is dropped on its own. The compiler is all-or-nothing, but the composition validates each override individually first, keeps the valid ones, and logs each drop with its reason. A ceiling violation on one tenant therefore cannot revert every other tenant to the template amount, which would be a silent cap increase. An amount of zero or less is rejected too, since it would switch the cap off rather than lower it.

What a counter is kept under

Two caps that look independent can share one counter, and one cap can fan out into hundreds. Which happens is decided by the counter’s identity, and that identity is the limit id, the node, the values of the dims the limit constrains, the period, and the algorithm. The amount and the mode settings are not part of it.

• Two different limit ids do not share a counter, even at the same node.
• The same limit at two nodes has two counters. That is what makes a template a per-node cap rather than a shared one.
• Editing an amount keeps the counter and the usage it has accumulated. Changing period: or algorithm: rotates it to a fresh cell, so usage so far is not carried across.
• A limit with match: { model: "gpt-*" } gets one counter per distinct requested model. gpt-4o and gpt-4o-mini are separate cells, not one shared bucket.

The dim vocabulary is closed at three: model, provider, endpoint. The model dim is the model the client asked for, not the one that served the request, so a route that rewrites gpt-4o to a cheaper tier still counts under gpt-4o. A key’s model scope is a separate check, re-asserted against the served model.

Model patterns go through Go’s path.Match, where * does not cross a /. So llama* does not match accounts/fireworks/models/llama-v3; write accounts/fireworks/models/llama* instead. The same rule governs a key’s model scope, so one mis-written glob can both under-select a counter and over-restrict a key.

Shapes that resolve to nothing

Three declarations load, validate and compile, and then stop nothing. Nothing at runtime tells you either: tokentriage govern preview prints all three as though they were live.

attach: { level: global } is the same trap under a different name. Config validation admits global as a level, but no tree node is ever created at that level, so the template fans out to nothing. The spelling that works is attach: { node: __global }, which is permitted without tree membership.

An rpm or concurrency limit carrying match.model or match.provider cannot fire either, for a different reason. Those two classes are evaluated at the first gate, before the request body is parsed, so the model and provider dims are empty strings at that moment and the match fails against them. match.endpoint is fine on both, and match.model on tpm is fine too, because a tpm limit is evaluated at the second gate, after the body has been read. What governance does not do collects these with the rest of the disclosed gaps.

What this page does not cover

What each limit type counts, when its counter can overshoot, and which store it can bind to belong to The six limit types and How a limit counts. Who may write an override, and how far down the tree that permission reaches, is Delegated self-serve. What a tenant spent, as opposed to what they were allowed to spend, is Cost analytics.