# Configure, test and promote a gateway Open a gateway's draft, add a model and a streaming route, inspect which one a request takes, save it safely, then release it. Source: https://tokentriage.com/docs/build/gateway-builder --- import Callout from '../../../components/Callout.astro'; import Details from '../../../components/Details.astro'; import NextSteps from '../../../components/NextSteps.astro'; A cheaper model for streaming requests is a candidate, not a decision. In this tutorial we'll take one gateway from that idea to a deployed release: add the model, branch on whether the caller asked for a stream, check which branch a request would take without calling a provider, and save the whole thing under the revision you read. The artefact you end up looking at is one `gateway-config/v1` document with two model choices, two routes and a pinned policy revision, and a release id that pins it. This tutorial takes the command-line path, because it is the one that shows the document. The gateway's **Configure** and **Test** tabs in the portal drive the same operations over the same draft, so each step below ends by naming what the portal calls that step and where the two differ. ## What you need | Thing | Needed? | Why | |---|---|---| | an application with one LLM gateway | yes | [Create your company and first application](/docs/get-started/first-company) makes both | | `config.edit` on that application | yes | reading a draft needs `gateway.read`; saving one needs `config.edit` | | `deploy` in the target environment | for the last step only | a protected environment also needs `deploy.protected` | | a saved provider connection for the environment | before any paid step | [Add a saved provider connection](/docs/build/provider-connections) | | a provider API key of your own | no | you bind a connection by id; the key stays with TokenTriage | | a config file on disk | no | the draft is a document the control plane stores | Steps 1 to 7 make no provider call, so you can walk them before anybody has added a key. **No time budget is published for this path.** These steps have not been walked end to end against a deployment, so there is no measured number to print, and a guessed one would be worse than none. Every code block below carries the same note for the same reason. The commands are `tokentriage cloud`, which needs `--base-url` and reads the management key from `TOKENTRIAGE_MANAGEMENT_KEY`. Each example omits the repeated `--org`/`--workspace`/`--app` flags after the first. ## 1. Open the draft The draft is the gateway's own document in one environment, and reading it is how you get the revision number every later save has to echo. ```console $ tokentriage cloud draft get --base-url https://app.example.com \ --org org_… --workspace t_… --app app_… \ --gateway gw_… --env env_… --json > draft.json ``` That command has not been run against the private deployment; its verbs and flags come from the CLI's own definitions. A gateway nobody has edited yet is seeded on that read: one model choice on the single eligible inventory target, and one default route on the baseline policy, so the first thing you edit is a working configuration rather than an empty file. Seeding writes a row, so it happens only for a caller who also holds `config.edit`. You know it worked when `draft.json` carries a `document` whose `schema_version` is `gateway-config/v1`, and an `expected_revision` you can read. A gateway with nothing eligible, or with several eligible targets and no way to choose, comes back with `saved: false`, a warning and revision `0`: a seed that guessed would put your money behind a model nobody picked.
You hold `gateway.read` and not `config.edit`, so there is nothing to read and nothing this call may create. That answer is the truth rather than a permission error: the draft does not exist yet. Ask for `config.edit` on the application, or have somebody who holds it open the gateway once.
**Availability.** Landed. ## 2. Add a model choice Add the candidate before the route, so the branch you write next has something to point at. A model choice names an entry of the operator inventory. Read the inventory first, because it carries a verdict per target and the reason when one cannot be used. ```console $ tokentriage cloud gateway targets --gateway gw_… --env env_… ``` That command has not been run against the private deployment. Then add the choice to `draft.json`. The id is yours to mint: `mc_` plus twelve base58 characters, unique inside this document. ```json { "id": "mc_7Qd2fLm9Xk3T", "label": "Economical classifier", "target_id": "tgt_…", "settings": { "max_output_tokens": 512 } } ``` That fragment is a request document, not captured output; it follows the field list in the [gateway configuration reference](/docs/reference/gateway-configuration). Adding a model makes it available to the draft. It sends nothing to the provider and changes nothing that is running. You know it worked at the next save: a target this environment may not use is declined `422 inventory_target_ineligible` naming the target, rather than accepted and discovered later. Adding a model also adds a slot to bind. A binding is keyed on the target's upstream, not on the model, because several models of one upstream share one credential. The document carries one `connection_bindings` entry per upstream, and a slot nobody bound is what `422 connection_slot_unbound` reports when you create a release. **Availability.** Landed. In the portal this is the Configure tab's **Model choices** list, which offers only the targets the inventory calls eligible, and its **Provider connections** list, which shows one row per upstream with either the bound connection or **No connection bound**. Adding a model there sends nothing to the provider and binds no connection. ## 3. Add a conditional route, and keep the default A gateway with one model needs no routes. You add one when two kinds of request should be treated differently and the answer has to be predictable. The one condition v1 registers is `request.stream`, a boolean saying whether the caller asked for a streamed response. It is always present, so the branch cannot be skipped by a missing value. A route pins a policy revision rather than a policy, so read the revision you mean to pin. ```console $ tokentriage cloud policy get --policy rtp_… --revision 4 ``` That command has not been run against the private deployment. Then write the routes. Order is the semantics: first match wins, and the unconditional route is last. ```json [ { "id": "rt_5Hb8xNq2Wme1", "label": "Streaming", "predicate": { "fact": "request.stream", "op": "is", "value": "true" }, "policy_id": "rtp_…", "policy_revision": 4, "model_choice_id": "mc_7Qd2fLm9Xk3T" }, { "id": "rt_3Jc9yPr4Znf2", "label": "Everything else", "predicate": null, "policy_id": "rtp_…", "policy_revision": 4, "model_choice_id": "mc_1Ab2cDe3Fgh4" } ] ``` Those are request documents rather than captured output. Routes resolve in four steps, and each step can only narrow the answer. 1. Evaluate the conditional routes in document order. 2. Select the first route whose predicate matches. Its pinned policy chooses the model. 3. If no predicate matched, select the route whose predicate is `null`. 4. Execute only that route's own retry and fallback behaviour. Step 4 is the one readers get wrong. A sibling route does not run because a provider call failed; a fallback belongs to the selected route and is visible in `fallback_model_choice_id`. You know it worked when a save is accepted. Three shapes are declined by name: no unconditional route, an unconditional route that is not last, and two routes with the same predicate. **Availability.** Landed. `request.stream` is the whole registered fact set for v1, so treat the reference table as the list rather than an example. ## 4. Reuse a shared policy, or copy it Three things you can do to a policy look similar and differ in who else they affect, which is why this step comes before you touch one. | Action | What it does | Who else is affected | |---|---|---| | publish a revision | appends a new revision to the shared policy | nobody immediately; each consumer stays pinned until its own draft adopts it | | copy the policy | starts an independent policy at that revision's content | nobody, now or later | | adopt a revision | repoints one named draft at a revision | nobody else; that draft still has to be released | Pinning is what stops your edit from rerouting a colleague's traffic. Before publishing, read who holds the revision you are changing. ```console $ tokentriage cloud policy impact --policy rtp_… --revision 4 $ tokentriage cloud policy publish --policy rtp_… --file policy.json \ --expected-revision 4 $ tokentriage cloud policy adopt --policy rtp_… --revision 5 \ --gateway gw_… --env env_… --expected-revision 3 ``` Those three commands have not been run against the private deployment. The impact report is filtered to scopes you may read, and consumers outside them appear as counts rather than names. Use `tokentriage cloud policy copy --policy rtp_… --revision 4 --display-name "…"` instead when your change should affect nothing else. You know it worked when the draft's route names the revision you adopted. A pin on a revision nobody published, or one in another company, is the same answer: `404 policy_revision_not_found`. **Availability.** Landed. ## 5. Edit a protection component An inherited rule you cannot see is the most common cause of "the builder says one thing and the gateway does another", so read the palette before editing a card. ```console $ tokentriage cloud gateway components --gateway gw_… ``` That command has not been run against the private deployment. Each card carries its kind, its placement, its bounds, its fields with types, which fields are inherited and from where, and its docs path. The palette is filtered rather than annotated: a card this gateway's profile cannot execute, or one whose action you do not hold, is absent, so a tool iterating the list cannot offer one by accident. Protection itself is an attachment in the document, pinned to a profile revision, with an optional list of inherited rules turned off here. ```json { "profile_id": "grp_…", "profile_revision": 2, "disabled_rule_ids": ["inherited-pii-monitor"] } ``` That is a request fragment rather than captured output. A required company or workspace rule is not in that list's reach. Mandatory bindings come first in each phase, and where two rules disagree the more restrictive action wins, ordered `block`, `modify`, `monitor`, `simulate`. You know it worked when the next inspect shows the rule in the phase you expected, with the mandatory ones you did not touch still there. **Availability.** The palette is landed. The guardrail composition it feeds lands with the release-derivation increment. The portal's Configure tab is one-way here: it renders each stage's cards and lists the inherited rules this draft already has switched off, with a **Switch back on** control for each. It offers no control for switching an inherited rule off, because no customer operation publishes a profile's rule ids and the page would have no id to send. Write `disabled_rule_ids` from the command line, as above, until that read exists. ## 6. Inspect which route a request would take This is the step that costs nothing. Inspect compiles the subject in memory, evaluates one described request through the routing engine, and contacts no provider, so a broken reference or an unreachable branch surfaces before anything is charged. Send the subject and the request facts to `customer.gateways.inspect`. ```json { "subject": { "kind": "saved_draft_revision", "draft_revision": 4 }, "request": { "stream": true, "model": "…" } } ``` That body has not been sent against the private deployment. `kind` is `saved_draft_revision` or `applied_release`, and it has to agree with the identity beside it. The answer names the matched route, the policy revision it selected, the tier and the fallback plan, plus highlights that map each decision back to the card that caused it. It returns no model answer, because no model was called. You know it worked when a `stream: true` request matches your streaming route and a `stream: false` request matches the default one. If both match the default, the predicate value is a string in this document: `"true"`, not `true`. Inspect answers "which route and which model", never "is the answer any good". One live response is not evidence that a cheaper model preserves quality; that takes the same candidate against many examples. The portal's **Test** tab offers **Inspect routing** without a provider call. The CLI addresses the same operation: ```console $ tokentriage cloud gateway inspect --base-url https://app.example.com \ --org org_… --workspace t_… --app app_… --gateway gw_… \ --env env_… --draft-revision 4 --json ``` Use `--release rel_…` instead to inspect an applied release. Export the management key as `TOKENTRIAGE_MANAGEMENT_KEY`; the command does not need browser cookies. The key's owner still needs access to the gateway and environment. The shipped `sdk/ts/examples/iterate_routing.ts` and `sdk/python/examples/iterate_routing.py` execute read, CAS save, inspect and destination promotion review. They add a model choice and a **first non-streaming branch** using the default route's pinned policy; they refuse a draft that already has that branch. Choose a fresh canonical model-choice ID (`mc_` followed by 12 base58 characters). A saved choice must be referenced by a route before it can be released. The programs stop after review. Call their exported `iterateRouting` / `iterate_routing` function with explicit destination bindings and `confirmPromote: true` / `confirm_promote=True` to apply. An unbound slot remains a blocker. A first deployment has no head row: only the exact `404 not_found` after successful destination review becomes expected head revision zero; other refusals remain errors. {/* Executed through actual PostgreSQL, production compiler and cookieless management-key HTTP in TestS06InspectPromotionExamplesAgainstRealCompiler. No provider execution; connection metadata is an explicitly declared fixture. This is local integration evidence. */} ## 7. Compare Draft with Applied Before any paid call, settle which of the two configurations you are reading. Applied is what the environment serves now; the draft is what you are preparing. `customer.gateways.diff` takes two subjects and returns the change set. ```json { "from": { "kind": "applied_release", "release_id": "rel_…" }, "to": { "kind": "saved_draft_revision", "draft_revision": 4 } } ``` That body has not been sent against the private deployment. The answer groups changes the way the document does: model choices, routes with their old and new positions, protections by placement, and bindings by slot. It carries both effective digests and an `identical` flag, so "no difference" is an answer rather than an empty screen. You know it worked when the routes group names your new route as added and the digests differ. Identical digests with a changed document means you diffed the draft against itself. The diff operation is available through the SDK and customer HTTP lane. The portal's **Changes** panel answers two questions and keeps them apart. "Not saved yet, in this browser" is the editor's own comparison of what you have typed against what you opened, because no server operation can be asked about bytes it has never seen. "The saved draft, against what is applied" is this diff, computed by the control plane, which is why a reordered route reads "moved" and a re-pinned policy reads "re-pinned" instead of both reading "changed". ## 8. Save it, and recover from a stale draft A save replaces the draft with the document you send, and nothing is merged into it. That is what makes "read it, edit the file, send it back" mean what it looks like. ```console $ tokentriage cloud draft put --gateway gw_… --env env_… \ --file draft.json --expected-revision 3 ``` That command has not been run against the private deployment. `--expected-revision` is the draft revision you read in step 1, not an internal counter. If somebody saved while you were editing, the save comes back `412 draft_revision_stale` carrying the winner's revision and their whole document, so you can show the conflict without a second request. The same fence is in the TypeScript client, where that answer arrives as a typed error. ```typescript import { CustomerClient, StaleDraftError } from "tokentriage/customer"; const client = new CustomerClient({ baseUrl: "https://app.example.com", managementApiKey: process.env.TOKENTRIAGE_MANAGEMENT_KEY ?? "", }); try { await client.gateways.saveDraft(org, ws, app, gw, env, { document, expected_revision: 3, }); } catch (err) { if (!(err instanceof StaleDraftError)) throw err; // err.headRevision is the winner's revision; err.latest is their document. showConflict(err.headRevision, err.latest); } ``` That snippet has not been run against the private deployment; it uses the shipped `gateways.saveDraft` signature and the `StaleDraftError` fields. Recovering means rebasing, not retrying: apply your change to the winner's document, then send it with their revision as `expected_revision`. Both versions stay in the append-only history, which `tokentriage cloud draft history` reads, so neither edit is lost. You know it worked when the response carries a revision above the one you sent. Deleting a model choice a route still selects is declined `409 model_choice_referenced` with the routes listed. **Availability.** Landed. The portal's **Save this draft** panel states the revision it was built on, and a lost race offers **Rebase onto revision N** built from the refusal body. Pressing it adopts the winner's document whole and says in as many words that your unsaved edits are not in it, because a three-way merge of somebody else's routing decisions would be a guess about traffic. ## 9. Create a release, then deploy it These are two decisions and two grants, which is the vocabulary this step exists to teach. `customer.releases.create` pins the configuration under `config.edit`: the settings revision, the derived policy and profile revisions, the mandatory generation, every slot's connection, and the effective digest. Creating one changes nothing about what the gateway serves. `customer.deployments.deploy` makes one release the desired one under `deploy`, echoing the deployment head's revision as `expected_revision`, or `0` for a target nobody has deployed. A protected environment also needs `deploy.protected`. Cells observe the head and the state moves to applied. Two refusals belong to the create rather than to the first live request: a compiled artifact with no unconditional route is `422 default_route_missing`, and one dispatching to a slot nothing resolves is `422 connection_slot_unbound`. You know it worked when the deployment head names your release as both desired and applied, and one permitted request through that environment returns a request id whose record names the same release. If it does not, roll that environment back to the prior release. **Availability.** Release, deployment and promotion operations are declared in the lane and land across this private test. A deployment with no release ledger behind them answers `503` with "deployments not configured", so a wired path and a misspelled one stay distinguishable. [Move a gateway between environments](/docs/build/environments-and-promotion) covers promotion, the deploy gate and rollback. The portal reaches the same place by a different route, and the difference is worth knowing before you look for a button. There is no create-a-release panel and no deploy panel on the Configure tab. What it offers is **Promote this configuration**, which reviews against a destination environment and then applies, and applying is what derives the release and deploys it. The tab's **Applied** view is a read of the release ledger rather than a second canvas: it shows the release's identity, which is the revisions it pins, its effective digest, its required-controls generation and the connections it binds. It draws no routes and no model choices, because the release API publishes neither, and it says so on the panel rather than redrawing the draft under an "Applied" label. ## What this tutorial does not cover - How a policy picks a model inside the route you selected: [How a routing decision is made](/docs/build/routing-decision). - Writing a policy's content: [Routing policy language](/docs/build/policy-language). - Every field, status and operation named here: [Gateway configuration reference](/docs/reference/gateway-configuration). - Running a question through the draft against a provider, and keeping the answer where your team can review it: [Test a gateway draft and save the run for your team](/docs/build/test-and-share-gateway-runs).