# Read your gateway's traffic and spend Ask a hosted gateway what it served, what it cost and how much of the answer is missing, from the CLI or either SDK. Source: https://tokentriage.com/docs/build/gateway-insights --- import Callout from '../../../components/Callout.astro'; import CodeTabs from '../../../components/CodeTabs.astro'; import Details from '../../../components/Details.astro'; import NextSteps from '../../../components/NextSteps.astro'; Your application has been sending requests through a hosted gateway for a day. You want four things back: how many requests it made, how they ended, how long they took, and what the upstreams charged for them. Three clients ask that question and read the same document: the `tokentriage cloud` CLI, the TypeScript SDK and the Python SDK. This page asks it once in each, then spends most of its length on the awkward parts of the answer: a cell with no figure in it, a table that was capped, a saved view this build can no longer answer. You need an organization id (`org_…`), a workspace id (`t_…`), a management key in `TOKENTRIAGE_MANAGEMENT_KEY`, and `metadata.read` on the scope you ask about. Any money figure also needs `cost.read`. There is no database to stand up and no browser session: these reads go to the API the portal itself uses. [Who can do what in your company](/docs/build/organizations-and-access) covers how a grant reaches a person. One request at a time is a different read: `tokentriage cloud request get --request lreq_…` (and `client.requests.get(org, ws, req)` in either SDK) answers what served that single request — gateway, environment, release or saved draft revision, purpose, the acting credential by label, the token usage, the cost with its pricing state, what the guards did, and, for a hosted test, the run, case and arm it is evidence for. A request id from another workspace is answered "not found" rather than "forbidden", and a cost nobody has priced yet reads `unpriced` rather than $0.00. This page does not cover saving a view, editing one, or sending app feedback. Those are write operations, and they have their own pages: [Save a gateway view and share it](/docs/build/save-gateway-views) and [Report and read app feedback](/docs/build/report-app-feedback). [How to read a cost figure](/docs/build/cost-figures) covers where a dollar amount comes from in the first place; [Gateway insight reads](/docs/reference/gateway-insights) is the field-by-field contract behind this page. Every block below is program output. The commands ran against a loopback server replaying `sdk/spec/insight-read-fixtures.json`, the recording the lane's own handler wrote, so `--base-url` and the request line name `127.0.0.1` and a port that changes per run. Yours name your deployment. ## Set the overview population Open a gateway's **Overview** and choose its environment, time window, activity purpose, and release. Application traffic is the default; playground tests, comparison runs and judge calls are separate choices. Each choice changes the overview, previous-period comparison, time series, release markers, model/route/release facets and feedback coverage together. A saved question retains that same population. The release menu uses releases returned for the selected gateway and environment. It shows at most the first 100 available choices and discloses when more exist. **All releases** still queries the complete authorized population; choosing a release narrows the server query, rather than filtering the displayed page. Changing environment clears the previous environment's release selection. If release choices cannot be read, the page explains that failure and retains the current filter. A chart's **View as table** button exposes its underlying measured values and missing-value reasons. It works with the keyboard. A facet opens the Requests tab with its time, purpose and release context; any additional filter applied only to a bounded page is labeled as such. Missing coverage or an unpriced population remains visible after filtering. A smaller population does not establish savings or repair a journal gap. #### Read the current workspace budget The overview's **Workspace budget** card reads the enforcing authority separately from the traffic charts. Check its enforcement state, current reset window, limit, committed capacity, and remaining capacity. Change the chart dates without treating the budget as a historical balance. Use **Refresh budget** for a fresh authority read. Committed capacity includes escrow; it is not settled provider spend. The authority does not separately report pending and reserved amounts, so those read **Unknown**. Missing access or a failed read does not mean there is no cap. The card requires workspace `cost.read`, the same permission as the supported budget API. ### Review the attention cards The overview shows at most four cards derived from the displayed evidence: the largest observed model spend, requests that did not complete, response p95 above your display threshold, and missing price or comparison quality evidence. Enter an optional threshold in milliseconds below the charts. It applies to this open view only; it does not configure an alert or change routing. No latency card appears until you set a positive threshold. Use **Latency distributions** to read exact p50 and p95 timings with their sample counts. Expand **Separate outcome populations** to compare successful, refused, failed, cancelled, and partial requests. Cancellation comes from the recorded delivery outcome and is counted separately from failure. Each timing shows observed and eligible requests. First-output timing applies to streams; an ordinary request supplies no first-output sample. A missing duration stays unavailable, while an explicitly recorded 0 ms remains a measurement. Older records without duration provenance can make a population partial. Its percentiles describe only observed samples. Follow each card's link to inspect the figures behind it. Partial costs can change the spend ranking, and incomplete coverage can move a percentile in either direction. Missing cost is never zero. The quality card reflects the selected saved comparison; changing scope or selecting an incomplete run removes any claim of complete evidence. ## Read saved cost and quality evidence In **Overview**, choose a saved comparison and a terminal run. The card reads the saved run's results, including the sample version, both configuration pins, the acceptance rule and its quality counts. Baseline, candidate and judge spend remain separate from the sample's measured cost reduction. A running run shows its status until you refresh. The card also checks whether the recorded baseline is still the applied release. A changed baseline leaves the historical measurements readable and labels their applicability. A failed read offers a retry; changing the environment clears evidence from the previous scope. For **Application traffic**, the same selected window and release can supply a qualified cohort projection. A cohort is the set of requests that match the saved comparison's frozen input and configuration conditions. Read its covered and uncovered counts, tested pairs, assumptions and price basis beside the projected difference. A negative difference means a projected cost increase. It is not realized savings or a monthly forecast. Other activity purposes do not request an application projection. Missing qualification, unpriced requests or an unavailable reader produce a reason, not a zero. The sample's observed spend remains a different measurement. The [comparison guide](/docs/build/compare-cost-and-quality#read-the-evidence) shows the read-only CLI and SDK calls for the same evidence. ## Run a scoped read against your deployment The shipped `sdk/ts/examples/read_gateway_insights.ts` and `sdk/python/examples/read_gateway_insights.py` read an overview, then one bounded page of requests and attempts under the **same server-issued read context**. Set `TOKENTRIAGE_BASE_URL`, `TOKENTRIAGE_MANAGEMENT_KEY`, `TOKENTRIAGE_ORG`, `TOKENTRIAGE_WORKSPACE`, `TOKENTRIAGE_APPLICATION`, `TOKENTRIAGE_GATEWAY`, `TOKENTRIAGE_ENVIRONMENT`, `TOKENTRIAGE_FROM` and `TOKENTRIAGE_TO`. The last two are a half-open UTC window. Set optional `TOKENTRIAGE_RELEASE` to narrow all three example reads to one release; the CLI equivalent is `--release rel_…` (repeatable to select several releases). The examples preserve the complete answer, including numeric strings, coverage, cursors and truncation; they do not add missing groups or convert unknown costs to zero. Neither example writes or executes a provider request. Both programs and the shipped CLI were executed against the private customer API on September 23, 2026, for an existing one-minute A1 population. The overview and both drill-downs agreed on two requests, two primary attempts and 306 tokens. The two priced request receipts sum to 2,685,000 nano-USD ($0.002685), matching the overview, model/release facets and hourly bucket. Those answers still disclosed `ingest_gap:tt-prod-pool`; the exact visible rows do **not** establish that the whole source is complete. This is a read-only reconciliation, not new provider traffic. {/* Receipt: docs/manual-testing/foundation/S09/customer-reads-20260923v/README.md. The older fixture transcripts below retain their original provenance and are not restamped as live output. */} ## Ask for an overview An overview answers one value per metric over one half-open UTC window, grouped by activity purpose. Purposes stay in separate groups: a production request and a playground experiment are two populations, and their sum describes neither. `--purpose application` is the default. ```console $ tokentriage cloud insights overview \ --metric requests.v1 --metric provider_spend.v1 \ --metric cost_per_1000_requests.v1 --metric tokens.v1 \ --metric response_latency.v1 \ --org org_K11111111111 --workspace t_K11111111111 \ --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z \ --base-url http://127.0.0.1:2091 application requests.v1 1284 count application provider_spend.v1 unavailable (unpriced_population) application cost_per_1000_requests.v1 unavailable (unpriced_population) application tokens.v1 84210773 tokens (FLOOR: ingest_gap:gateway_journal) application response_latency.v1 412 milliseconds ``` 1,284 requests, and no spend figure. That pairing is the subject of the next section. Three lines went to stderr in the same run, so `--json | jq` keeps working and the qualifications still arrive: ```text cloud: customer.insights.query POST http://127.0.0.1:2091/portal/api/v1/customer/v1/organizations/org_K11111111111/workspaces/t_K11111111111/insights/query cloud: coverage is incomplete for this window (ingest_gap:gateway_journal); measured values use an incomplete population; counts/sums are lower bounds, ratios/percentiles may move either way cloud: gateway_journal is behind (produced 918290, indexed 918273), so the most recent traffic may be missing ``` The second and third lines are the answer describing its own gaps: some journal rows for this window have not been indexed yet, so the token figure is a floor, and the producer is 17 sequence numbers behind at the moment of the read. The same question from either SDK. `insights.query` takes the query document, and the answer arrives under the envelope's `data`: ```python answer = client.insights.query(org, workspace, { "schema": 1, "shape": "overview", "from": "2026-09-20T12:00:00Z", "to": "2026-09-21T12:00:00Z", "purposes": ["application"], "metrics": ["requests.v1", "provider_spend.v1", "cost_per_1000_requests.v1", "tokens.v1", "response_latency.v1"], })["data"] for group in answer.get("groups", []): for cell in group["cells"]: value = cell["value"] if value["state"] == "unavailable": print(cell["metric"], "is unavailable:", value["reason"]) elif value["state"] == "partial": print(cell["metric"], value["number"], value["unit"], "FLOOR", value["coverage"]) else: print(cell["metric"], value["number"], value["unit"]) ``` ```typescript const { data: answer } = await client.insights.query(org, workspace, { schema: 1, shape: "overview", from: "2026-09-20T12:00:00Z", to: "2026-09-21T12:00:00Z", purposes: ["application"], metrics: ["requests.v1", "provider_spend.v1", "cost_per_1000_requests.v1", "tokens.v1", "response_latency.v1"], }); for (const group of answer.groups ?? []) { for (const { metric, value } of group.cells) { if (value.state === "unavailable") { console.log(`${metric} is unavailable: ${value.reason}`); } else if (value.state === "partial") { console.log(metric, value.number, value.unit, "FLOOR", value.coverage); } else { console.log(metric, value.number, value.unit); } } } ``` Both print the same five rows, one per cell, differing only in how each language renders a list of strings: ```text requests.v1 1284 count provider_spend.v1 is unavailable: unpriced_population cost_per_1000_requests.v1 is unavailable: unpriced_population tokens.v1 84210773 tokens FLOOR ['ingest_gap:gateway_journal'] response_latency.v1 412 milliseconds ``` Every cell names its unit, and the unit is one of the eight the contract carries — `response_latency.v1` answers in `milliseconds`, the spelling the catalogue publishes for it. A formatter may be keyed off the unit; a ninth spelling cannot reach it. ## Read a cell that has no number in it A measure comes back in one of three states, and a renderer that treats them alike is how a bill gets under-reported. | state | carries | read it as | |---|---|---| | `available` | `number` and `unit` | a figure this service stands behind | | `partial` | `number`, `unit` and `coverage` | a floor: the truth is larger, for the listed reasons | | `unavailable` | `reason`, and nothing else | there is no figure to plot; the reason says what to do | An `unavailable` cell has no `number` and no `unit` on it at all, so there is nothing for a renderer to reach for and format as `$0.00`. "Nothing was spent" and "we cannot tell you what was spent" are different facts, and printing the second as the first understates a bill by an unknown amount. The overview above is `unavailable{unpriced_population}`: rows exist, and some of them carry no price, so a sum over them would be smaller than the spend. Wait for pricing to land, and the cell fills in on its own. A different reason on the same cell needs a different person. Reading it as somebody who holds `metadata.read` and not `cost.read`: ```console $ tokentriage cloud insights overview \ --metric requests.v1 --metric provider_spend.v1 \ --metric response_latency.v1 \ --org org_K11111111111 --workspace t_K11111111111 \ --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z \ --base-url http://127.0.0.1:2096 application requests.v1 1284 count application provider_spend.v1 unavailable (scope_not_authorized) application response_latency.v1 412 milliseconds ``` The read itself went through. Only the cell that would disclose money is withheld, and the reason names the remedy: ask an administrator for `cost.read`, rather than waiting for a figure that is not coming. Treat every cell's reason separately, and do not collapse an absent cell into a failed request. A window in which nothing happened is a third thing again. There are no groups to iterate, and the CLI says so in a sentence: ```console $ tokentriage cloud insights overview \ --metric requests.v1 --metric provider_spend.v1 \ --org org_K11111111111 --workspace t_K11111111111 \ --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z \ --base-url http://127.0.0.1:2099 no traffic in this window for the purposes asked; this is an ABSENCE of rows and not a measurement of zero ``` In either SDK the same answer is a `data` document with no `groups` member: `"groups" in answer` is false while `read_context` is still there to continue from. Synthesizing a row of zeros at that point publishes a measurement nobody made. ## Read a money figure without losing digits When every row in the window is priced, the spend cell fills in, and the CLI prints dollars beside the integer it was computed from: ```console $ tokentriage cloud insights overview \ --metric requests.v1 --metric provider_spend.v1 \ --metric cost_per_1000_requests.v1 \ --org org_K11111111111 --workspace t_K11111111111 \ --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z \ --base-url http://127.0.0.1:2094 application requests.v1 1284 count application provider_spend.v1 $1234.567890123 (1234567890123 nano_usd) application cost_per_1000_requests.v1 $0.961500693 (961500693 nano_usd) ``` Money travels as a decimal string of integer nano-USD, and the string is the figure. Keep it as a string, or move it into an integer type that is not floating point. Nano-USD passes what a double holds without loss at about $9,007,199 of spend, which a large customer reaches inside a month; from there, two spends one nano apart compare equal. ```python spend = answer["groups"][0]["cells"][1]["value"] print(spend["number"], spend["unit"], type(spend["number"]).__name__) print("exact:", int(spend["number"]) * 2) # 1234567890123 nano_usd str # exact: 2469135780246 ``` ```typescript const spend = (answer.groups ?? [])[0]!.cells[1]!.value; if (spend.state === "available") { console.log(spend.number, spend.unit, typeof spend.number); console.log("as a BigInt:", BigInt(spend.number) * 2n); } // 1234567890123 nano_usd string // as a BigInt: 2469135780246n ``` ## Drill into the requests behind a number A drill-down returns the rows themselves. `--grain request` is one row per logical request, with retries and fallbacks folded into the request they belong to; `--grain attempt` is one row per dispatch, including the fallbacks that were discarded. No prompt text, no completion text and no money travel at either grain. ```console $ tokentriage cloud insights drilldown --grain request \ --org org_K11111111111 --workspace t_K11111111111 \ --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z \ --base-url http://127.0.0.1:2101 lrq_K11111111111 2026-09-21T11:22:00Z completed gpt-4o-mini 2140ms first-output 318ms attempts 2 lrq_M11111111111 2026-09-21T11:29:00Z completed claude-haiku 884ms first-output - attempts 1 lrq_N11111111111 2026-09-21T11:48:00Z refused - 41ms first-output - attempts 0 ``` Read the second row's `first-output -` as an absence. That request did not stream, so it produced no time-to-first-output sample; `0ms` would have made it the fastest row in the table and pulled every first-output quantile down with it. The third row is a refusal before any upstream was dispatched, which is why it has no model and no attempts, and it still counts as one logical request. One line went to stderr, and it changes what the table means: ```text cloud: this answer was TRUNCATED at a row bound; it is a page of the rows and not all of them ``` A capped table presented as a whole one is the table somebody makes a decision from. In the SDKs the same fact is `answer["truncated"]`, and the rows carry `first_output_known` rather than a sentinel time: ```python for row in answer["requests"]: first = f"{row['first_output_ms']}ms" if row["first_output_known"] else "-" print(row["logical_request_id"], row["status"], row["total_ms"], first) print("truncated:", answer.get("truncated", False)) # lrq_K11111111111 completed 2140 318ms # lrq_M11111111111 completed 884 - # lrq_N11111111111 refused 41 - # truncated: True ``` ```typescript for (const row of answer.requests ?? []) { const first = row.first_output_known ? `${row.first_output_ms}ms` : "-"; console.log(row.logical_request_id, row.status, `${row.total_ms}ms`, first); } console.log("truncated:", answer.truncated ?? false); // lrq_K11111111111 completed 2140ms 318ms // lrq_M11111111111 completed 884ms - // lrq_N11111111111 refused 41ms - // truncated: true ``` ## Find out what you are allowed to ask The catalogue lists every metric this deployment publishes, what unit it answers in, whether reading it needs `cost.read`, and whether anything writes it yet. It is answered from the server's build and reaches no warehouse, so it works identically where the analytics producer is switched off, which makes it the thing to build a metric picker from. ```console $ tokentriage cloud insights catalog --org org_K11111111111 \ --base-url http://127.0.0.1:2103 requests.v1 count produced metadata.read How many requests did this application make? provider_spend.v1 nano_usd produced metadata.read + cost.read What did this application's upstream usage cost? cost_per_1000_requests.v1 nano_usd produced metadata.read + cost.read What does a thousand requests cost? tokens.v1 tokens produced metadata.read How many tokens did the upstreams read and write? response_latency.v1 milliseconds produced metadata.read How long did a request take end to end? ``` Those are five of the twenty-four rows that run printed; the full listing, and the closed enums that follow it on stderr, are in [Gateway insight reads](/docs/reference/gateway-insights). A row whose `producer_status` reads `no_producer` is nameable today and answers `unavailable (no_producer)` until its slice lands, so a picker can show it disabled rather than pretend it is missing. ## Open a saved view somebody kept A saved view stores a question. It stores no SQL and no grant, so opening one re-resolves the reader's own scope and answers under that: a view shared with a team shows each member what they may read. ```console $ tokentriage cloud view list --org org_K11111111111 \ --workspace t_K11111111111 --base-url http://127.0.0.1:2105 sview_K11111111111 Checkout spend, last 24h team fixed overview revision 1 sview_M11111111111 A question this build lost team fixed overview revision 1 ``` `view get` prints the stored question without answering it. `view open` returns both the question and the answer in one document, so the chart and the caption above it cannot come from two different moments of your access: ```console $ tokentriage cloud view open --org org_K11111111111 \ --view sview_K11111111111 --base-url http://127.0.0.1:2109 sview_K11111111111 Checkout spend, last 24h team fixed overview revision 1 application requests.v1 1284 count application provider_spend.v1 unavailable (unpriced_population) application cost_per_1000_requests.v1 unavailable (unpriced_population) application tokens.v1 84210773 tokens (FLOOR: ingest_gap:gateway_journal) application response_latency.v1 412 milliseconds ``` In the SDKs the opened document splits into `view` and `answer`, and the answer's cells are the ones from the section above: ```python opened = client.saved_views.open(org, "sview_K11111111111")["data"] view, answer = opened["view"], opened["answer"] print(view["name"], view["audience"], view["time_intent"], view["revision"]) # Checkout spend, last 24h team fixed 1 ``` ```typescript const { data: opened } = await client.savedViews.open(org, "sview_K11111111111"); const { view, answer } = opened; console.log(view.name, view.audience, view.time_intent, view.revision); // Checkout spend, last 24h team fixed 1 ``` ## When the read is turned away Three refusals reach a caller of these operations, and they send you to three different places. Keep them apart. | What you see | What happened | What to do | |---|---|---| | `HTTP 403`, `type=forbidden` | the scope names something your bindings do not reach | ask for the grant; retrying changes nothing | | `HTTP 503`, `type=coverage_unknown` | the ingest side cannot bound what is missing for this window | retry once the frontier has moved | | `HTTP 503`, `type=dependency_unavailable` | the analytics service could not be reached for this read | retry, and quote the request id | | `HTTP 422`, `type=validation_failed` | a saved view asks something this build no longer publishes | edit the view's question | The 403 and the 503 look nothing alike in a terminal, which is the point of separating them: a surface that printed one sentence for both would send a customer to an administrator during an outage. ```text customer.insights.query was refused (HTTP 403); detail: foundation: forbidden: metadata.read is not permitted at application:app_M11111111111; type=forbidden; request_id=req_forbidden_foreign_application customer.insights.query was refused (HTTP 503); detail: the ingest side cannot bound what is missing for this window; retry once the frontier has moved; type=coverage_unknown; request_id=req_coverage_unknown_503 ``` Both SDKs make the same split into two classes, and carry the remedy the server's own catalogue publishes for each code: ```text forbidden: InsightForbiddenError 403 forbidden refresh_access unavailable: InsightUnavailableError 503 coverage_unknown retry ``` The 422 is the odd one. A view whose stored question this build cannot re-ask still reads perfectly through `view get`, and fails on `open`: ```text customer.saved_views.open was refused (HTTP 422); detail: foundation: unsupported combination: saved view sview_M11111111111 asks a question this build cannot re-ask: insightcontract: metric version unknown; type=validation_failed; request_id=req_saved_view_open_stale_question ``` It is not a 404 and not a 403: the view is there, and you are allowed to read it. The remedy is to edit the question.
`validation_failed` is in the lane's published table with `next_action: check_selection`, so both SDKs raise the BASE class for it — `InsightRefusalError` — rather than `InsightForbiddenError` or `InsightUnavailableError`. It is neither a permission problem nor an outage: catching the base class gets you the published remedy, and `type` still reads `validation_failed`.
## What these reads do not answer Three metrics in the catalogue carry `producer_status: no_producer`: `test_quality.v1`, `paired_reduction.v1` and `attention_rules.v1`. Naming one in `--metric` is valid, and the cell answers `unavailable (no_producer)` rather than a zero, so a dashboard can show the question and say what is missing. `budget_standing.v1` carries `producer_status: produced_elsewhere`. Read its current control-plane standing through the budget operation; generic insight queries do not produce that standing. `saved_view.v1` carries `producer_status: produced_elsewhere`: the saved views themselves are recorded, by the saved-view operations rather than by the analytics projection, so they are read through those operations and not from an insight query. `cohort_projection.v1` also carries `producer_status: produced_elsewhere`. Its frozen comparison rule and qualified application counts are composed by the comparison results operation described above. Generic insight queries do not produce that estimate or grant access to comparison evidence. Two spend figures on this product are computed over different populations and must not be subtracted from each other. A paired comparison's **measured sample reduction** is arithmetic over the tested cases, on the baseline and candidate configurations, in the window the test ran. **Provider spend** here is settled cost on production traffic, over the window you asked about, on whatever release was deployed. Different window, different rows, different baseline. A reduction measured on a sample is not money that left your account, and neither figure becomes the other by subtraction. Saving, editing and deleting a view are write operations this page leaves out, not ones the clients lack: `cloud view save|edit|delete` takes the question as a file, and both SDKs carry `create`, `update` and `delete`. Only the portal can save the question that is on a screen, because it is the surface that has one. App feedback is a different credential again: a runtime key at the gateway, so neither the CLI nor the customer-API client carries it. Charts belong to the portal, not to this API. And a drill-down never returns prompt or completion text: the tables behind it hold none. Partial counts and additive spend/token sums are lower bounds. Ratios (including cost per request) and percentiles computed from incomplete populations can move higher or lower as missing rows arrive; they are not lower bounds. A producer with no indexed observation has **unmeasured freshness**, even if both reported counters are zero.