# Report and read app feedback Send one thumbs-up or thumbs-down from your application for a request a gateway served, and read how much traffic it covers. Source: https://tokentriage.com/docs/build/report-app-feedback --- import Callout from '../../../components/Callout.astro'; import CodeTabs from '../../../components/CodeTabs.astro'; import Details from '../../../components/Details.astro'; import NextSteps from '../../../components/NextSteps.astro'; Your users already know which answers were useful. Your gateway does not, because latency and token counts say nothing about whether the model was right. App feedback closes that gap with one rating per request: a person pressed thumbs-up or thumbs-down, and your application forwards it. The rating is filed against the logical request the gateway already recorded, so it joins the traffic you can already measure, and the coverage card in the portal tells you how much of that traffic anybody rated. Ratings ride the same credential and the same door as your inference calls: a `ttv_` runtime key holding `feedback.write`, at your assigned gateway's base URL. There is no management key on this path and no portal step. What you need from the response you are rating is its logical request id, which both SDKs read out of the response metadata. For a reporter that only sends feedback, mint a separate credential with an explicit `actions: ["feedback.write"]` selector. It cannot invoke models. Omit `actions` to keep the normal inference-only default; combine `inference:invoke` and `feedback.write` explicitly only when the same application credential needs both. Do not combine an explicit action list with the legacy `observations` option. ```console tokentriage cloud credential create --base-url "$CUSTOMER_BASE" \ --org "$ORG" --service "$SERVICE" --env "$ENV" --gateway "$GATEWAY" \ --label "Feedback reporter" --action feedback.write ``` Keep the one-time secret private. Set `TT_RUNTIME_KEY` from your secret store, then submit the seven-field JSON document using the assigned gateway URL: ```console tokentriage cloud feedback submit --gateway-base "$GATEWAY_BASE" \ --gateway "$GATEWAY" --env "$ENV" --body-file rating.json ``` The command makes one request and does not retry automatically. For a duplicate, resubmit the identical document with its original idempotency key. For a correction, use the next revision and a fresh idempotency key. Read coverage with `cloud insight overview`; feedback submission itself grants no analytics read access. This page covers the hosted gateway lane. The self-hosted ledger has its own `ledger.feedback` call with a different credential and a different shape, documented in [outcomes and feedback](/docs/sdks/outcomes). Both SDK blocks below were run against the recorded gateway answers in `sdk/spec/feedback-write-fixtures.json`, replayed by a loopback server; the printed values are that recording, and the Python transcript is the one captured. The TypeScript call is copied from `sdk/ts/test/unit/gateway_feedback.test.ts`, which replays the same cases. ## Send one rating A submission is seven fields and there is no way to send an eighth: the server rejects unknown members, and every scope member (account, organization, application, environment, workspace, gateway) is turned away by name. Where the rating is filed is stamped from the key that was admitted, so your application cannot reach it. ```python receipt = gateway.submit_feedback( logical_request_id="lrq_01J9Z2K5Q7X3M8N4P6R1T2V5W7", issuer="end-user-7f3k", feedback_id="answer_quality", value="up", revision=1, idempotency_key="idem_01J9Z2K5Q7X3M8N4P6R1T2V5W8", occurred_at=datetime(2026, 9, 21, 12, 0, 0, tzinfo=timezone.utc), ) print(receipt.accepted, receipt.duplicate, receipt.current, receipt.seq) # True False up 18446744073709551615 ``` ```typescript const receipt = await gateway.submitFeedback({ logicalRequestId: "lrq_01J9Z2K5Q7X3M8N4P6R1T2V5W7", issuer: "end-user-7f3k", feedbackId: "answer_quality", value: "up", revision: 1, idempotencyKey: "idem_01J9Z2K5Q7X3M8N4P6R1T2V5W8", occurredAt: new Date("2026-09-21T12:00:00.000Z"), }); ``` Each field carries one decision: | Field | What it decides | |---|---| | `logical_request_id` | which served request this is about | | `issuer` | who is rating: an end user, a reviewer, a support agent | | `feedback_id` | which question they answered, so two questions about one answer stay apart | | `value` | `up` or `down`, and there is no neutral | | `revision` | 1 for a first rating, 2 for a correction, up to 5 | | `idempotency_key` | makes a retry of the same rating a no-op | | `occurred_at` | when the person rated, in UTC with milliseconds | A rating with no neutral is a choice you inherit. An unrated request is absent from the population rather than sitting in the middle of it, which is what lets the coverage figures below mean anything. ## Retry without double-counting The receipt is the ledger's own statement. `seq` is a 64-bit counter carried as a decimal string, because the largest value it can hold does not survive a double, and `match: pending` says the join to the request happens at query time rather than now. Send the identical rating a second time and it comes back as a success that changed nothing: ```text accepted: False duplicate: True current: up revision: 1 stored_revisions: 1 seq: 18446744073709551615 str match: pending ``` That script printed the type of `seq` beside it: `str`, because the client keeps the counter as a string and parses it into no number of its own. A duplicate raises nothing in either SDK. `accepted` false with `duplicate` true is idempotency working: your caller's intent is already durable. Treating it as a rejection is how a retry loop turns one rating into a queue nobody drains. ## Let somebody change their mind A correction is the next revision of the same `(logical_request_id, issuer, feedback_id)`, not a second rating. One issuer may revise up to five times inside a seven-day window from the request, and the gateway admits 60 ratings a minute per service. The highest stored revision wins on read, whatever order the revisions arrived in. A revision that arrives late is a slow network and does not overwrite a newer verdict, and `current` on each receipt tells you which value now wins. Two shapes are refused before anything leaves your process, which keeps a typo out of your retry budget: ```text class: ConfigurationError code: feedback_value_invalid message: value must be "up" or "down". There is no neutral rating: an unrated request is absent from the population, which is a different fact from a middle score. class: ConfigurationError code: feedback_revision_invalid message: revision counts from 1 per (request, issuer, feedback_id) and must be a positive integer. A correction is the NEXT revision. ``` A sixth revision is not one of them: the client sends it and the gateway answers 412, because whether the revisions are spent is the server's fact and not the client's. ## Tell the six refusals apart Each status is a different answer, and collapsing any pair would tell a caller to retry something permanent or to abandon an outage. The Python classes are below; TypeScript raises the same names. | Status | Class | Reason code | What to do | |---|---|---|---| | 401 | `AuthenticationError` | `account_key_inactive` | the runtime key was revoked; issue a replacement | | 403 | `PermissionDeniedError` | `feedback_forbidden` | the key does not hold `feedback.write` | | 409 | `ConflictError` | `feedback_conflict` | that revision is stored with a different verdict; use the next revision | | 412 | `PolicyDeniedError` | `feedback_precondition_failed` | outside the seven-day window, or the five revisions are spent | | 422 | `ValidationError` | `feedback_invalid` | the body carried a member the gateway stamps itself | | 429 | `RateLimitError` | `feedback_rate_limited` | wait `retry_after_seconds`, which the gateway sends | | 503 | `ServiceUnavailableError` | `feedback_unavailable` | retry; a retry of the same rating is a no-op | The 429 is the only one carrying a wait, and the client reports what the gateway sent rather than inventing a backoff: ```text class: RateLimitError code: feedback_rate_limited status: 429 retry_after_seconds: 1 message: Too many ratings from this runtime key. Retry shortly. ```
The document is well formed; the platform's own rule turned it away. A caller who classified it as validation would go looking for a bad field, and there is not one. The sentence names both causes because the client cannot tell which applied without reading state it does not hold.
## Read how much traffic anybody rated The gateway Overview carries a feedback coverage card, one block per activity purpose, because a production request and a playground experiment are two populations. Coverage divides **distinct rated requests** (`app_feedback_rated_requests.v1`) by **eligible requests** (`requests.v1`). Both use requests admitted in the selected window. Two people rating one of ten requests means one rated request and **10% coverage**, not 20%. Replayed feedback and revisions cannot increase that count. A valid rating may arrive after the selected window closes. It counts toward the window in which its request was admitted, up to the read's evaluation time and within the seven-day feedback limit. A rating for a request admitted before the window does not increase this window's coverage. Refresh to include newly indexed feedback. **Ratings received in this window** is a separate count, `app_feedback.v1`. It counts latest verdict identities `(logical_request_id, issuer, feedback_id)` by feedback time, so it can exceed the number of rated requests. It is never used as the coverage numerator. The card keeps both counts and the eligible population visible. When no request in the cohort has feedback, the card says **No feedback yet**. Missing measurements stay unavailable. If eligible requests are only a floor because capture is incomplete, the card shows the counts without a share: dividing by that floor would overstate coverage. Missing feedback is not a positive result. A window with a 4% rated share has told you about 4% of the traffic and nothing whatever about the rest, and the same holds for a window with no ratings at all. The card also splits ratings by where they came from and never merges the sources: an application's users, a member of staff reviewing an answer, and a judge model scoring one are three different claims. On this build only the app-reported row has a producer, so it is the only row that renders. The positive and negative halves render as named unavailable reasons for the same cause: nothing publishes that split yet, and inferring it from a count would invent the one number a team would act on. Each group offers a drill-down button, "Show eligible requests", which opens the request list for that purpose over the same window. The list is the group's eligible requests; this build publishes no per-request verdict to filter it by, so read it as the population the figures were taken over. ## Read one request's feedback Reading verdicts back uses the `feedback` grain of `customer.insights.query`, which is registered. A deployment older than this change refuses the grain: it answers `400 invalid_query` or `422 validation_failed`, and the CLI and both SDKs report that as "not served by this deployment". Every block marked "Not yet executed against a deployment" replays a recorded answer from `sdk/spec/insight-read-fixtures.json`; none was captured from a running deployment. The coverage card counts rated requests. When somebody complains about one answer, you want the verdicts on that one request: who rated it, which way, at which revision, and whether the rating has joined the request yet. This is a management-lane read, not a gateway call. It needs a management key holding `metadata.read` on the scope. The `ttv_` runtime key that submits ratings cannot make it, and `feedback.write` grants no read. The issuer's own label also needs `content.read`, as [the issuer handle](#tell-issuers-apart) explains. A verdict is windowed on when it was given, its `occurred_at`, in a half-open UTC window `[from, to)`. To read everything one request can ever receive, start the window at the request's `admitted_at` from its receipt and end it at the earlier of now, rounded up to the minute, and `admitted_at` plus seven days and one minute. The portal does not show a request's app feedback yet. A feedback section on the request receipt is planned for a later release; until it arrives, use the CLI or an SDK. ### From the CLI `--grain feedback` with `--request` scopes the page to one request on the server: _Not yet executed against a deployment._ ```console $ tokentriage cloud insights drilldown --grain feedback \ --request lrq_K11111111111 \ --org org_K11111111111 --workspace t_K11111111111 \ --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z cloud: customer.insights.query POST http://127.0.0.1:18701/portal/api/v1/customer/v1/organizations/org_K11111111111/workspaces/t_K11111111111/insights/query lrq_K11111111111 fis_fedcba9876543210fedcba9876543210 fb-7c1e up rev 1 pending app-reported lrq_K11111111111 fis_0123456789abcdef0123456789abcdef fb-2a90 down rev 3 matched app-reported lrq_K11111111111 unassigned fb-0001 up rev 1 matched app-reported cloud: some verdicts were stored before issuer handles existed and show issuer `unassigned`; rows of one issuer may not group together cloud: gateway_journal is behind (produced 918290, indexed 918273), so the most recent traffic may be missing ``` Each line is one verdict: request, issuer, `feedback_id`, value, revision, match state, and `app-reported`. The `cloud:` lines went to stderr. That run used a CLI built from source against a loopback server replaying the recorded case `drilldown_feedback_request`, so the `127.0.0.1` address is that server's, passed with `--base-url` and left out of the command line shown; you see your own base URL. The watermark line comes from the recorded read context, whose gateway journal had not finished indexing. A key that also holds `content.read` gets the issuer's label in the second column, replayed from `drilldown_feedback_content`: _Not yet executed against a deployment._ ```text cloud: customer.insights.query POST http://127.0.0.1:18702/portal/api/v1/customer/v1/organizations/org_K11111111111/workspaces/t_K11111111111/insights/query lrq_K11111111111 svc.search-ranker fb-7c1e up rev 1 pending app-reported lrq_K11111111111 user-4821 fb-2a90 down rev 3 matched app-reported lrq_K11111111111 legacy:user-7 fb-0001 up rev 1 matched app-reported cloud: some verdicts were stored before issuer handles existed and show issuer `unassigned`; rows of one issuer may not group together cloud: gateway_journal is behind (produced 918290, indexed 918273), so the most recent traffic may be missing ``` `--match` reads one match state across the window instead of one request. This replays `drilldown_feedback_unmatched`: _Not yet executed against a deployment._ ```console $ tokentriage cloud insights drilldown --grain feedback --match unmatched \ --org org_K11111111111 --workspace t_K11111111111 \ --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z cloud: customer.insights.query POST http://127.0.0.1:18703/portal/api/v1/customer/v1/organizations/org_K11111111111/workspaces/t_K11111111111/insights/query lrq_P11111111111 fis_0123456789abcdef0123456789abcdef fb-9d02 down rev 1 unmatched app-reported lrq_Q11111111111 fis_fedcba9876543210fedcba9876543210 fb-4410 up rev 2 unmatched app-reported cloud: gateway_journal is behind (produced 918290, indexed 918273), so the most recent traffic may be missing ``` `tokentriage cloud request get --feedback` reads the receipt, then makes the feedback read as a second call over that request's admission window, at the request's own purpose, and prints it under **App feedback**. Against a deployment older than this change, which refuses the grain, replayed from `drilldown_feedback_invalid_filter` after a recorded receipt, it prints: _Not yet executed against a deployment._ ```console $ tokentriage cloud request get --org org_ABCDEFGHJKLM --workspace t_ABCDEFGHJKLM \ --request lreq_2026_09_20_000001 --feedback cloud: customer.requests.get GET http://127.0.0.1:18705/portal/api/v1/customer/v1/organizations/org_ABCDEFGHJKLM/workspaces/t_ABCDEFGHJKLM/requests/lreq_2026_09_20_000001 request lreq_2026_09_20_000001 status succeeded served by gateway gw_ABCDEFGHJKLM, environment env_ABCDEFGHJKLM, release rel_ABCDEFGHJKLM purpose application (chat, streaming) acted as runtime_credential A1 production key attempt att_2026_09_20_000001 primary via openai/gpt-4o-mini usage 812 in, 240 out, 128 cached tokens (provider_reported) cost unpriced (no price has been settled for this attempt yet) guards clean note: this request has no cost figure (unpriced (no price has been settled for this attempt yet)). It is NOT $0.00. App feedback cloud: customer.insights.query POST http://127.0.0.1:18705/portal/api/v1/customer/v1/organizations/org_ABCDEFGHJKLM/workspaces/t_ABCDEFGHJKLM/insights/query not served by this deployment: it refused the feedback grain (HTTP 400 invalid_query) Error: app feedback not served by this deployment: it refused the feedback grain (HTTP 400 invalid_query): customer.insights.query was refused (HTTP 400); detail: insightcontract: invalid document; type=invalid_query; request_id=req_drilldown_feedback_invalid_filter ``` The receipt lines come from a recorded receipt, and both addresses are the replay server's. The receipt above the section stands. The command still exits 1, so a script that asked for the feedback knows it did not get it. Each refused read names its own cause: | Status | The CLI says | What to do | |---|---|---| | 403 | `app feedback denied: this key may not read app feedback in this scope (an administrator grants metadata.read)` | ask an administrator for `metadata.read`; a retry does not help | | 400, 422, 501 | `app feedback not served by this deployment: it refused the feedback grain (HTTP 400 invalid_query)` | nothing to retry: this deployment predates the grain | | 503 | `app feedback unavailable: the service could not answer the feedback read now; retry later (HTTP 503 dependency_unavailable); this is not a permission problem` | retry later | The status and slug in parentheses are the server's own. ### From Python and TypeScript Both SDKs send the read through `insights.query` with `drilldown: "feedback"` and a `feedback_filter`. The examples `sdk/python/examples/observability_request_diagnosis.py` and `sdk/ts/examples/observability_request_diagnosis.ts` make it after the receipt read, with the window computed from `admitted_at` by `feedback_window` and `feedbackWindow`. _Not yet executed against a deployment._ ```python import os from tokentriage.customer import (CustomerClient, InsightForbiddenError, InsightRefusalError, InsightUnavailableError) client = CustomerClient(os.environ["TOKENTRIAGE_BASE_URL"], os.environ["TOKENTRIAGE_MANAGEMENT_KEY"]) try: answer = client.insights.query("org_K11111111111", "t_K11111111111", { "schema": 1, "shape": "drilldown", "drilldown": "feedback", "page_size": 50, "from": "2026-09-20T12:00:00Z", "to": "2026-09-21T12:00:00Z", "purposes": ["application"], "feedback_filter": {"logical_request_id": "lrq_K11111111111"}, })["data"] except InsightForbiddenError as error: # 403: a permission, not an outage print("denied", error.status, error.code) except InsightUnavailableError as error: # 503: retry later print("unavailable", error.status, error.code, error.next_action) except InsightRefusalError as error: # 400/422: this deployment does not serve the grain print("not served", error.status, error.code, error.next_action) else: for row in answer.get("feedback", []): print(row["issuer_ref"], row.get("issuer", "-"), row["feedback_id"], row["value"], "rev", row["revision"], row["match"], row.get("purpose", "-")) print("coverage:", answer.get("coverage", [])) ``` ```typescript import { CustomerClient, InsightForbiddenError, InsightRefusalError, InsightUnavailableError, type InsightAnswer, } from "tokentriage/customer"; const client = new CustomerClient({ baseUrl: process.env.TOKENTRIAGE_BASE_URL!, managementApiKey: process.env.TOKENTRIAGE_MANAGEMENT_KEY!, }); try { const answer = (await client.insights.query("org_K11111111111", "t_K11111111111", { schema: 1, shape: "drilldown", drilldown: "feedback", page_size: 50, from: "2026-09-20T12:00:00Z", to: "2026-09-21T12:00:00Z", purposes: ["application"], feedback_filter: { logical_request_id: "lrq_K11111111111" }, })).data as InsightAnswer; for (const row of answer.feedback ?? []) { console.log(row.issuer_ref, row.issuer ?? "-", row.feedback_id, row.value, "rev", row.revision, row.match, row.purpose ?? "-"); } console.log("coverage:", answer.coverage ?? []); } catch (error) { if (error instanceof InsightForbiddenError) { console.log("denied", error.status, error.code); // 403: a permission } else if (error instanceof InsightUnavailableError) { console.log("unavailable", error.status, error.code, error.nextAction); // 503 } else if (error instanceof InsightRefusalError) { console.log("not served", error.status, error.code, error.nextAction); // 400/422 } else { throw error; } } ``` The Python program ran with the in-repo SDK (`sdk/python/src`) under Python 3.14.5, three times, against a loopback server replaying `drilldown_feedback_request`, `drilldown_feedback_content` and `drilldown_feedback_invalid_filter` in turn. It printed: ```text fis_fedcba9876543210fedcba9876543210 - fb-7c1e up rev 1 pending - fis_0123456789abcdef0123456789abcdef - fb-2a90 down rev 3 matched application unassigned - fb-0001 up rev 1 matched application coverage: ['issuer_ref_unassigned'] fis_fedcba9876543210fedcba9876543210 svc.search-ranker fb-7c1e up rev 1 pending - fis_0123456789abcdef0123456789abcdef user-4821 fb-2a90 down rev 3 matched application unassigned legacy:user-7 fb-0001 up rev 1 matched application coverage: ['issuer_ref_unassigned'] not served 400 invalid_query check_selection ``` The TypeScript program has not been run. Its calls mirror `sdk/ts/test/unit/customer_insights.test.ts`, which replays the same three cases. ### Read the three match states Every row carries `match`, the state of its join to the request it names: | `match` | Means | |---|---| | `matched` | the request was found in the reader's scope; the row carries its `purpose` | | `pending` | the request may not be indexed yet, so the row may still become `matched` | | `unmatched` | the gateway journal is indexed past the point the request would have arrived, and no such request was served in the scope the rating was filed under | `pending` is "not yet", and `unmatched` is "no such request", not "deleted". While the gateway journal's progress is unknown, a row stays `pending` rather than turning `unmatched`. A rating is filed in the scope of the key that submitted it, so the recorded `pending` row above, filed under another application, joins no request there. ### Newest revision wins A row is one verdict identity (request, issuer, `feedback_id`) at its newest revision, whatever order the revisions arrived in. `revision` is your application's own number, carried as a decimal string, and `value` and `occurred_at` are that revision's. A row with a revision above 1 was revised. There is no count of earlier revisions: storage may already have folded them together, and a count would overstate the trail. Reading changes nothing about writing. A `pending` row is not a reason to resend a rating: the [retry rules above](#retry-without-double-counting) still apply, and resending the identical document with its original idempotency key comes back as a duplicate that changed nothing. ### Tell issuers apart `issuer_ref` is on every row: an opaque handle, `fis_` followed by 32 lowercase hex characters, stored the first time the workspace sees an issuer and not derived from the label. Key your rows on it. A verdict stored before handles existed shows `unassigned`, and the answer then carries the coverage token `issuer_ref_unassigned` once; rows of one such issuer may not group together. `issuer`, the label your application sent, is added only when the reader also holds `content.read`, on the same rows and the same handles. The CLI prints the label when the server sent one and the handle otherwise. ### App-reported, and which purpose a row belongs to Every row carries `source: app_reported`, which the CLI prints as `app-reported`: your application said so, and the gateway did not observe it. A `matched` row carries its request's `purpose` and is filtered by the `purposes` the read asked for. A `pending` or `unmatched` row has joined no request, so its purpose is unknown: it is included whatever `purposes` you asked for, with `purpose` absent. Do not count those rows toward a purpose. ## What this lane does not carry There is no `tokentriage cloud feedback` command, and the customer-API client has no feedback operation in either SDK: a management key holding `feedback.write` would be a wide credential doing a narrow credential's job. No deployment runs the `feedback` grain yet, so the aggregate counts on the card are the only served read surface in production; the registered read is [above](#read-one-requests-feedback). A rating carries no free-text note, and nothing on this path reads a rating back out through the gateway. A score your own system produces about a request, such as a resolved support ticket, is not a rating. [Report outcomes about your requests](/docs/build/reported-outcomes) describes the reported-outcome operations for that claim.