Test a gateway draft and save the run for your team

You changed a routing rule and you want to know what the gateway now answers, before anyone’s users find out. Typing the question into a chat window somewhere else tells you what that window’s model says. It tells you nothing about your compiled configuration, your guardrails, your budget or the connection your slots bind.

The playground runs your question through the gateway itself: the same compiled document, the same protection stages, the same admission and budget checks, the same saved provider connection. Choose scratch or team capture before dispatch. Team capture turns the spot check into a saved run colleagues can read, review, and cite when they deploy.

This page is the task. Look at what a test would do, run it, follow it, read it, and leave a verdict on it. For the configuration you are testing, start with Configure, test and promote a gateway.

What you need

Two words carry the whole page. A case is one question, stored under the gateway and versioned, so the same question can be asked of staging and of production. A run is one execution of one case version in one environment, which is what makes it evidence about that environment and nothing else.

Look before you spend

Every paid control on this lane sits under the same five facts, and the read that produces them calls no provider and costs nothing. Ask for them first:

$ tokentriage cloud tests inspect --base-url https://app.example.com \
    --org org_acme00000001 --workspace t_acme0000001 --app app_userportal01 \
    --gateway gw_ticketclass1 --env env_staging00001 --draft-revision 7
# … two `cloud:` diagnostic lines naming the scope and the operation URL
subject   saved_draft_revision 7 (digest sha256:4f1ac0f0f4b2)
slot      primary pconn_9mQ2xR7tLbVd version 3: eligible
slot      fallback pconn_5kT8yH3wNcJr version 1: NOT ELIGIBLE (connection_purpose_not_granted)
ceiling   up to $0.05 (spend ceiling)
retention kept for 30 days (standard)
audience  everyone in this workspace with content.read

That capture came from the shipped command tree driven against a loopback server in the CLI’s own test suite, so the ids and the figures are fixtures rather than an invoice. Against your deployment the shapes are the same and the numbers are yours.

Read it as five answers:

A slot marked NOT ELIGIBLE is a run that will be declined once it starts. The slug names which door to open: connection_purpose_not_granted means the connection’s grant does not admit the playground purpose for this application and environment, so grant it on the connections page or bind one that already carries it.

The portal’s Test tab issues that same read when it opens, and prints the five facts above both buttons that spend money. You do not have to press anything to learn what pressing something would cost.

Choose scratch or keep, before anything runs

The choice between the two buttons is the product, and it is made before dispatch. No flag anywhere promotes a finished scratch run into a saved one; asking for one answers capture_choice_immutable.

Both spend money. The paid verbs put the preflight in front of you rather than trusting a sentence in a help screen:

$ tokentriage cloud tests run --gateway gw_ticketclass1 --env env_staging00001 \
    --draft-revision 7 --file prompt.txt
# … the same five preflight lines as above, on stderr
Error: `cloud tests run` was NOT sent: it causes a real provider call against a real saved credential, and the ceiling above is what it may cost. Re-run with --yes to send it (or with --json, which is the scripted path and skips this confirmation)

Add --yes to send it. --json also sends it directly, because a confirmation prompt inside a pipeline is a process that stops responding where nobody can see it, and cloud tests inspect --json answers the same question for nothing.

What a saved run records

A saved run is a row you can point a colleague at months later, so it keeps the parts that make the answer mean something:

• the case and the version it asked, pinned, so a question that moves later cannot rewrite what this run was about
• the input bytes as the server observed them, stored as a content object with its own retention class
• the subject: the configuration’s digest, its draft revision or release id, and the version of each connection it bound
• the result: a terminal state, the receipt id, the token counts, the latency, and the cost in integer nano-USD
• every verdict anyone recorded on it, in order, with the note each one carried

Nothing of a scratch run survives it, and that falls out of the shape rather than a policy: the scratch path writes no case, no run and no content object, so a later read finds nothing. Asking for a scratch answer after the event says scratch_output_not_retainable.

Follow the run to a terminal word

runs create answers 202 with a queued row and no bytes. A control-plane worker re-resolves your authority, mints a short-lived execution ticket and only then dispatches, which is why the answer is not in the reply. Poll the run:

$ tokentriage cloud runs get --gateway gw_ticketclass1 --env env_staging00001 \
    --run trun_3hF7kM2rTyWb --json
{
  "schema": "customer/v1",
  "request_id": "req_6bN4pS1vZxQe",
  "kind": "test-run",
  "data": {
    "run": {
      "run_id": "trun_3hF7kM2rTyWb",
      "state": "succeeded",
      "case_id": "tcase_2wK1nVQ8bdHp",
      "case_version": 1,
      "capture": "team"
    },
    "result": {
      "terminal_state": "succeeded",
      "cost_state": "priced",
      "cost_nano_usd": 41200,
      "output_ref": "cobj_8dL4vC6qEnHs"
    }
  }
}

State moves forward and stops. Six words, and the three terminal ones are alternatives rather than a sequence:

unknown_effect is terminal and is not a failure. The provider may have been called and charged. Check the receipt before running the case again, because running it again may be the second charge rather than the first.

Cost has three states of its own, and none of them is zero. priced carries the settled integer, pending means it has not settled yet, and unknown means this build cannot tell. A run whose cost is unsettled prints “Not settled yet”, not $0.00.

Read the answer, or the reason you cannot

A run and its text are two reads, and the second re-checks content.read at retrieval rather than at the reference. So a person who can open every run here can still be declined its prompt, and holding an object id is not holding the bytes:

$ tokentriage cloud content get --gateway gw_ticketclass1 --object cobj_8dL4vC6qEnHs
# … two `cloud:` diagnostic lines naming the scope and the operation URL
Error: customer.content.get was refused (HTTP 409); detail: scopedcontent: content purged: cobj_8dL4vC6qEnHs was purged on 2026-10-22T00:00:00Z; type=content_purged; request_id=req_1cV9jD5uXpKa

Content withheld is the portal’s name for that outcome: the run stands, its cost, its states and its verdicts read normally, and the text alone is unavailable. Which of the three it is matters, because they have different recoveries:

Collapsing the three into “unavailable” would tell a reader their prompt might come back when it cannot, and would hide the one case where asking a colleague works. An object addressed through a gateway that does not own it is a concealed 404, the same answer as one nobody ever wrote.

Review it, rerun it, share it

A saved run becomes evidence once somebody says what they think of it. Three operations close the loop, and each keeps the original intact.

Review records one verdict on a terminal run: approved, rejected or needs_work, with an optional note stored as its own content object under its own retention class. A later verdict supersedes an earlier one rather than editing it, so the history reads as a sequence of opinions instead of one mutable field. A verdict on a run that has not finished answers run_not_terminal; one written from a page that has gone stale answers review_superseded.

Rerun asks the same case version again and writes a new run with its own evidence. The subject is not copied from the earlier run, which is the part people get wrong: “the same as last time” is a configuration that may have moved since, so name --release or --draft-revision, or neither to use what the environment serves now. A rerun cannot change the capture choice its parent made.

Share answers with the run’s own path and two sentences about who can open it. It mints no token and publishes no URL. Whoever follows the link authenticates as themselves and reads what their own grants allow, which is why a colleague without content.read opens the run and not the prompt.

The same lane from a script

Both SDKs carry a worked example of the whole sequence: inspect, create the case, create the run, poll it, then read the bytes. The scripts stop before spending when a slot is ineligible, the spend ceiling is missing or malformed, or the published ceiling is above what the caller allowed. Their default limit is 50,000 micro-USD ($0.05); set TOKENTRIAGE_TEST_CEILING_MICRO_USD to a non-negative integer only when you intend to allow a different maximum. A missing cost figure is never treated as zero.

Use a management key in TOKENTRIAGE_MANAGEMENT_KEY, plus the explicit base URL, organization, workspace, application, gateway and environment variables shown in each example. No browser cookie is needed. These programs omit a subject selector and test what the environment currently serves. A management key cannot select a saved draft or an explicit release; use the authenticated browser playground for that workflow.

The input is a gateway request body, not plain prompt text:

{"model":"gateway","messages":[{"role":"user","content":"say hello"}],"max_tokens":64}

Python passes these JSON bytes to the SDK; TypeScript supplies their base64 encoding. The saved case retains exactly those request bytes. The programs print the run ID even when polling ends before completion; read that run again rather than restarting the program, which would create another paid run.

# sdk/python/examples/test_and_save_run.py, trimmed to the five calls
preflight = _data(client.tests.inspect(org, ws, app, gw, env, subject=plan.subject))
created = _data(client.cases.create(org, ws, app, gw, input=plan.input,
                                    title=plan.title, origin="authored"))
run = _data(client.runs.create(org, ws, app, gw, env, case_id=case_id,
                               case_version=case_version, capture="team"))
view = _data(client.runs.get(org, ws, app, gw, env, run_id))   # until terminal
content = _data(client.content.get(org, ws, app, gw, output_ref))

// sdk/ts/examples/test_and_save_run.ts, trimmed to the five calls
const preflight = data<TestPreflight>(
  await client.tests.inspect(org, ws, app, gw, env, plan.subject));
const created = data<CaseView>(
  await client.cases.create(org, ws, app, gw,
    { title: plan.title, origin: "authored", input: plan.input }));
const run = data<TestRun>(
  await client.runs.create(org, ws, app, gw, env,
    { case_id: caseId, case_version: caseVersion, capture: "team" }));
const view = data<RunView>(await client.runs.get(org, ws, app, gw, env, runId));
const content = data<ContentView>(await client.content.get(org, ws, app, gw, outputRef));

Every answer on this lane is {schema, request_id, kind, data} and the resource sits under data. The unit test that drives the TypeScript example through a fake transport asserts the request sequence, which is the part of an example that rots:

POST  …/environments/{env}/tests/inspect
POST  …/gateways/{gw}/tests/cases
POST  …/environments/{env}/tests/runs
GET   …/environments/{env}/tests/runs/run_1
GET   …/environments/{env}/tests/runs/run_1
GET   …/gateways/{gw}/tests/content/cobj_9

The same test pins what the script returns when the answer is withheld: state reads succeeded, output is null, and outputAbsentBecause carries the slug the server sent, such as content_read_denied. Neither example records a verdict, because a verdict belongs to a person, and a script that stamped “approved” on its own output would make the team’s history worthless.

What this lane does not do

Four boundaries, each of which someone has expected the other way round:

1. A saved run does not stream to you live. The worker owns the dispatch, so the run row moves queued to dispatched to a terminal word and you read the answer once it is stored. A scratch run answers synchronously, with the completion in the reply body.
2. A streamed answer is not schema-checked. format.jsonschema declares the output phase alone and no binding can name stream beside it, so a test that asks for a stream against an armed gateway arrives unvalidated, with stream_unsupported published on the protection read. Check a response against a JSON Schema has the detail.
3. An application key cannot choose what it runs against. A management key naming a subject answers app_key_cannot_select_subject with a 403. Selecting a draft revision or a release is something a person does while signed in; an application-class credential always gets what the environment is serving.
4. A deployment can be built without this lane at all. Then every row answers tests_not_enabled with a 501, by name rather than as a 404, so a missing feature and a mistyped path stay distinguishable. Nothing you did caused it and no retry changes it; whoever runs the deployment can turn it on.

Two neighbouring tasks live elsewhere. Comparing two configurations on one question, and judging quality rather than reading it, are not on this lane. What your company is billed for these runs, split from your product’s traffic, is on What your company is billed for.