# The tenancy hierarchy How org, team, user and key nodes resolve one request into a set of caps, and what an override can and cannot raise. Source: https://tokentriage.com/docs/build/governance-hierarchy --- import Callout from '../../../components/Callout.astro'; import Details from '../../../components/Details.astro'; import NextSteps from '../../../components/NextSteps.astro'; import GovernanceHierarchyDiagram from '../../../components/GovernanceHierarchyDiagram.astro'; import GovernanceResolveDiagram from '../../../components/GovernanceResolveDiagram.astro'; Three caps should hold on the same request at once: a monthly ceiling for the whole company, a tighter daily one for the team running the nightly batch, and a rate cap on one laptop so a runaway loop can't eat either. When one of the three stops that request, you want to know which one, and why the other two stayed quiet. All three hold because each request is resolved against a tree of tenancy nodes, and every cap the tree selects binds it. This page is the shape of that tree, the walk that turns it into a set of caps, and the one mechanism for giving a single node a bigger number than its neighbours. It assumes you know what the six limit types count; [The six limit types](/docs/build/governance-limits) is that page. What follows is about *which* caps apply to a request, not what each one measures. One example runs throughout: an org called `acme` with teams `ml` and `search`, two users under `ml`, and a virtual key held by `dev1`. ## The tree, and where it comes from A cap has to name who it applies to, and "a customer" is rarely one thing. It is a company, the teams inside it, the people inside those, and the credentials they hold. Governance models that as four levels (org, team, user, key), and you declare the first three. ```yaml governance: enabled: true # default false; an absent block is off tenants: # the declarative plane - org: acme teams: [ml, search] users: { ml: [dev1, dev2] } ``` That block materialises five nodes: `org:acme`, `org:acme/team:ml`, `org:acme/team:search`, `org:acme/team:ml/user:dev1` and `org:acme/team:ml/user:dev2`. Every team you name under `users:` has to be declared in `teams:` as well; a `users:` group naming a team that is not stops `tokentriage validate` with `governance.tenants[0].users["ml"]: names a team not declared in teams`. A node id is one or more `:` segments joined by `/`. The kind is lowercase letters; the id is lowercase letters, digits, `-` and `_`. Uppercase, spaces and a `..` segment are out of charset. That charset is checked where a delegated admin is authorised against a subtree, before the two ids are compared, so `org:acme/../org:evil` cannot be walked into another org's tree. It is not checked where a node or a key is created: `keys mint --owner` and the management API's node create both take the string as given, so a typo in either lands an identity whose chain resolves against nodes that do not exist. `__global` is the synthetic root. You cannot declare it in `tenants:` and it is not a tree node, but every chain ends at it, and a limit pinned there applies to everything the gateway serves, including traffic carrying no key. Nodes arrive from two places. `governance.tenants` is the declarative plane, edited in your config file and hot-reloadable. Nodes created through the management API are the runtime plane, stored in `governance.db`. At each arm both fold into one tree, so a runtime-created node can carry a config-declared template. When the same node id appears in both, the config spec is kept and the runtime row is skipped: no error, no warning. Creating a runtime node also has no effect on the live gate until the next governance reload. ## How a limit attaches You do not write a limit per tenant. A limit attaches once, in one of two forms, and the tree does the fan-out. - **Pinned** — `attach: { node: org:acme/team:ml }` gives one instance, at that node. - **A level template** — `attach: { level: team }` gives one instance per tree node at that level. Add a team to `tenants:` and it gets its own counter, with no edit to the limit. Config requires one of `node` or `level`, and only one; setting both or neither stops `tokentriage validate` with `governance.limits[0].attach: must set exactly one of {node, level}`. A pinned node has to be in the tree when governance compiles, and that one is not a validate-time check. `validate` accepts a pinned node that does not exist; boot is where it stops, with `governance: compile limits: govern: limit "ml_daily": pinned node "org:acme/team:nope" not in tenancy tree`. `__global` is the exception, permitted as a pinned node without tree membership, and that is the working spelling for a fleet-wide cap. ```yaml governance: limits: - id: acme_month # a template: one instance per org attach: { level: org } type: budget_usd amount_usd: 1000 period: month override: { allow: true, max_amount_usd: 5000 } - id: ml_daily # pinned to this node and no other attach: { node: "org:acme/team:ml" } type: budget_usd amount_usd: 200 period: day - id: user_rpm # a template: one instance per user attach: { level: user } type: rpm amount: 120 period: 1m - id: global_rpm # the spelling that reaches every request attach: { node: "__global" } type: rpm amount: 5000 period: 1m ``` That set passes `tokentriage validate` and compiles under `tokentriage govern preview`. None of the four declares a `posture:` block, and an omitted one means hard enforcement rather than off; [Turn governance on](/docs/build/governance-rollout) covers the ladder. Read the tree down the left and the limits down the right; the dashed edges are limits binding to a level. The key at the bottom is drawn as data rather than as a tree node, and that is accurate. A key is counted through the node it belongs to. ## How one request resolves A key can print its own chain, and that chain is the resolution walk. `GET /tt/v1/me` answers with it: ```console $ curl -H "Authorization: Bearer $TT_KEY" http://127.0.0.1:8790/tt/v1/me ``` ```json { "kid": "vk_FEKZdMs2UQRi", "node": "org:acme/team:ml/user:dev1", "chain": ["org:acme/team:ml/user:dev1/key:vk_FEKZdMs2UQRi","org:acme/team:ml/user:dev1","org:acme/team:ml","org:acme","__global"], "scopes": {"models":null,"providers":null,"endpoints":null,"allow_passthrough":false}, "limits": [ {"limit_id":"org_monthly_budget","type":"budget_usd","class":"spend","unit":"usd","node":"org:acme","period":"month","resets_at":"2026-09-01T00:00:00Z","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"limit_usd":{"usd":0.000001,"estimated":false,"unpriced":false},"observed_usd":{"usd":0.0000126,"estimated":true,"unpriced":false},"remaining_usd":{"usd":0,"estimated":true,"unpriced":false}}, {"limit_id":"user_concurrency","type":"concurrency","class":"count","unit":"concurrent-requests","node":"org:acme/team:ml/user:dev1","period":"","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"amount":10,"observed":0,"remaining":10}, {"limit_id":"team_rpm","type":"rpm","class":"count","unit":"requests","node":"org:acme/team:ml","period":"1m0s","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"amount":3,"observed":2,"remaining":1}, {"limit_id":"team_tpm","type":"tpm","class":"count","unit":"tokens","node":"org:acme/team:ml","period":"1m0s","suspended":false,"posture":{"mode":"on","enforce":"hard","fail":"open"},"amount":20000,"observed":0,"remaining":20000} ] } ``` `chain` carries five entries, deepest first. The first is the key leaf, formed as `/key:`. Then the three cumulative prefixes of the owner node. Then `__global`, where every chain terminates. The `limits` array is the caller's resolved set with each cap's observed and remaining figures; those four ids come from the sandbox the capture was taken on, not from the config above. Resolution turns that chain into a set of instances. Each step does one thing to the answer, and steps 1 to 6 read compiled metadata only, not a live counter. 1. **Walk the limits in ascending id.** The compiled order is lexical, so the order you wrote them in your config does not change the outcome. 2. **Drop a limit whose `match:` the request does not satisfy.** The test runs once per limit, before any node fan-out, and a dropped limit produces no instance and shows up in no probe trace. 3. **Expand each surviving limit into nodes.** A pinned limit yields its one node; a template yields every tree node at its level, in lexical order. 4. **Keep only the nodes in this request's chain**, together with `__global`. A node outside the chain is dropped here, so a limit sitting on another tenant's node contributes no instance to this request. 5. **Substitute the per-node override**, when one is stored for that limit and that node. It replaces the amount and nothing else. 6. **Order the surviving instances** by node depth descending, then limit id ascending, then node id ascending. 7. **Evaluate them in that order**, and stop at the first instance that is over its cap. Step 7 is a conjunction. Every selected instance binds this request, so an org cap binds every team and user beneath it. It binds them because the org's own instance is in the set, not because anything compares it against the team's. Every instance the chain selects binds the request, and the first one found over its cap stops it. The engine does not compare amounts across instances. The diagram puts four caps on that same chain and works down it, with `acme_month` lowered to $50 so the ordering is visible. The key leaf selects nothing. `user_rpm` at depth 3 is under its cap and admits. `ml_daily`, the team's $200/day budget, is over, so the request gets a 429 naming `ml_daily`. The org's $50 monthly cap is over too, and it is the tighter number, but it sits shallower and is not reached on this pass. The drawing flattens one thing. A rate cap is tested before the request body is read and a budget after it, so `user_rpm` and `global_rpm` are settled at the first gate and the two budgets at the second. Within a gate the order is the one drawn.
An anonymous request carries a chain of one entry, `__global`. Every template is therefore out of chain, and so is every limit pinned at a tenant node. The only caps that apply are the ones pinned at `__global` whose `match:` the request satisfies. Worth knowing before you leave `require_key` at `off` and expect an org template to hold the line.
## The tightest cap is not the one that names the 429 The operator guide that ships in the product repository, `docs/GOVERNANCE.md`, says resolution walks the chain as a conjunction and the tightest applicable limit wins. The first half is right. The second is not what the engine does. Amounts are not compared at any point. Instances are ordered by depth, then limit id, then node id, and evaluation stops at the first breach. So when a loose deep cap and a tight shallow cap are both over, the deep one is what lands in `refused_by` and in the `X-Tt-Refused-By` header. Tightest-wins holds in one weak sense: any breach stops the request, so the effective ceiling is the smallest applicable cap. What it misleads about is which limit gets named. That is the deepest one that was over, which can be the loosest. Read more than the one 429: `GET /tt/v1/me` lists every cap the caller resolves, so a shallower cap sitting at zero is visible even while a deeper one does the talking. ## Raising one node's cap One team needs a bigger budget than the template gives it, and forking the limit into a second definition leaves two things to keep in step. An override is the alternative: it changes the amount of one instance and leaves every other node on the template. The template declares the permission and the ceiling. `allow` defaults to `false` and `max_amount` defaults to `0`: ```yaml override: { allow: true, max_amount_usd: 5000 } ``` The config plane carries only that pair. The per-node override itself is a runtime-plane row, written through the management API and audited. `override: { allow: true }` with no `max_amount` or `max_amount_usd` declares no ceiling, and any positive amount is then accepted for any node: the comparison is skipped whenever the ceiling is `0`, which is what an omitted key leaves behind. Write the ceiling alongside `allow: true` every time. The opposite mistake is caught for you: a ceiling below the template's own amount stops `tokentriage validate` with `below the template amount_usd 1000 (an impossible ceiling)`. Three more things are worth knowing before you rely on one. **An override moves spend and quota only.** It is honoured by `budget_usd`, `quota_tokens` and `quota_requests`. On `rpm`, `tpm` and `concurrency` it is turned down with its reason, because those bake their amount into a per-window store or a live gauge when the limit compiles and do not read the instance's amount afterwards. The management API states the rule as data: ```json {"limit_id":"team_rpm","limit_type":"rpm","class":"count","unit":"requests","period":"1m0s","posture":"alert_only","attach":"template","level":"team","amount_count":2,"override_allowed":false} ``` To vary a rate cap per node, pin a separate limit at that node. **A runtime override does not enforce until the next governance reload.** The write is persisted and audited straight away, and it is disclosed as pending. The API says so itself: > definitions are the ARMED limits enforced now; overrides are persisted runtime writes > that take effect on the NEXT governance reload (compile-bound). No override is reported > as live-enforced here. The armed definition keeps showing the amount in force, so lowering a cap through the API cannot read as applied while traffic still flows against the old number. **One bad override is dropped on its own.** The compiler is all-or-nothing, but the composition validates each override individually first, keeps the valid ones, and logs each drop with its reason. A ceiling violation on one tenant therefore cannot revert every other tenant to the template amount, which would be a silent cap increase. An amount of zero or less is rejected too, since it would switch the cap off rather than lower it. ## What a counter is kept under Two caps that look independent can share one counter, and one cap can fan out into hundreds. Which happens is decided by the counter's identity, and that identity is the limit id, the node, the values of the dims the limit constrains, the period, and the algorithm. The amount and the mode settings are not part of it. - Two different limit ids do not share a counter, even at the same node. - The same limit at two nodes has two counters. That is what makes a template a per-node cap rather than a shared one. - Editing an amount keeps the counter and the usage it has accumulated. Changing `period:` or `algorithm:` rotates it to a fresh cell, so usage so far is not carried across. - A limit with `match: { model: "gpt-*" }` gets one counter per distinct requested model. `gpt-4o` and `gpt-4o-mini` are separate cells, not one shared bucket. The dim vocabulary is closed at three: `model`, `provider`, `endpoint`. The `model` dim is the model the client asked for, not the one that served the request, so a route that rewrites `gpt-4o` to a cheaper tier still counts under `gpt-4o`. A key's model scope is a separate check, re-asserted against the served model. Model patterns go through Go's `path.Match`, where `*` does not cross a `/`. So `llama*` does not match `accounts/fireworks/models/llama-v3`; write `accounts/fireworks/models/llama*` instead. The same rule governs a key's model scope, so one mis-written glob can both under-select a counter and over-restrict a key. ## Shapes that resolve to nothing Three declarations load, validate and compile, and then stop nothing. Nothing at runtime tells you either: `tokentriage govern preview` prints all three as though they were live. `attach: { level: key }` selects nothing in the shipped composition. The chain does carry a key leaf, but selection intersects the chain with the nodes that exist at that level, and no key-level node is ever created: the config plane emits org, team and user, and the runtime `nodes` table documents the same three kinds. The commented example in `configs/tokentriage.example.yaml` uses this shape. Attach at `level: user` instead, or pin the limit at the key holder's own node. `attach: { level: global }` is the same trap under a different name. Config validation admits `global` as a level, but no tree node is ever created at that level, so the template fans out to nothing. The spelling that works is `attach: { node: __global }`, which is permitted without tree membership. An `rpm` or `concurrency` limit carrying `match.model` or `match.provider` cannot fire either, for a different reason. Those two classes are evaluated at the first gate, before the request body is parsed, so the model and provider dims are empty strings at that moment and the match fails against them. `match.endpoint` is fine on both, and `match.model` on `tpm` is fine too, because a `tpm` limit is evaluated at the second gate, after the body has been read. [What governance does not do](/docs/build/governance-gaps) collects these with the rest of the disclosed gaps. ## What this page does not cover What each limit type counts, when its counter can overshoot, and which store it can bind to belong to [The six limit types](/docs/build/governance-limits) and [How a limit counts](/docs/build/governance-enforcement-classes). Who may write an override, and how far down the tree that permission reaches, is [Delegated self-serve](/docs/build/governance-self-serve). What a tenant spent, as opposed to what they were allowed to spend, is [Cost analytics](/docs/build/cost-analytics).