Report and read app feedback

Your users already know which answers were useful. Your gateway does not, because latency and token counts say nothing about whether the model was right.

App feedback closes that gap with one rating per request: a person pressed thumbs-up or thumbs-down, and your application forwards it. The rating is filed against the logical request the gateway already recorded, so it joins the traffic you can already measure, and the coverage card in the portal tells you how much of that traffic anybody rated.

Ratings ride the same credential and the same door as your inference calls: a ttv_ runtime key holding feedback.write, at your assigned gateway’s base URL. There is no management key on this path and no portal step. What you need from the response you are rating is its logical request id, which both SDKs read out of the response metadata.

For a reporter that only sends feedback, mint a separate credential with an explicit actions: ["feedback.write"] selector. It cannot invoke models. Omit actions to keep the normal inference-only default; combine inference:invoke and feedback.write explicitly only when the same application credential needs both. Do not combine an explicit action list with the legacy observations option.

tokentriage cloud credential create --base-url "$CUSTOMER_BASE" \
  --org "$ORG" --service "$SERVICE" --env "$ENV" --gateway "$GATEWAY" \
  --label "Feedback reporter" --action feedback.write

Keep the one-time secret private. Set TT_RUNTIME_KEY from your secret store, then submit the seven-field JSON document using the assigned gateway URL:

tokentriage cloud feedback submit --gateway-base "$GATEWAY_BASE" \
  --gateway "$GATEWAY" --env "$ENV" --body-file rating.json

The command makes one request and does not retry automatically. For a duplicate, resubmit the identical document with its original idempotency key. For a correction, use the next revision and a fresh idempotency key. Read coverage with cloud insight overview; feedback submission itself grants no analytics read access.

This page covers the hosted gateway lane. The self-hosted ledger has its own ledger.feedback call with a different credential and a different shape, documented in outcomes and feedback.

Both SDK blocks below were run against the recorded gateway answers in sdk/spec/feedback-write-fixtures.json, replayed by a loopback server; the printed values are that recording, and the Python transcript is the one captured. The TypeScript call is copied from sdk/ts/test/unit/gateway_feedback.test.ts, which replays the same cases.

Send one rating

A submission is seven fields and there is no way to send an eighth: the server rejects unknown members, and every scope member (account, organization, application, environment, workspace, gateway) is turned away by name. Where the rating is filed is stamped from the key that was admitted, so your application cannot reach it.

receipt = gateway.submit_feedback(
    logical_request_id="lrq_01J9Z2K5Q7X3M8N4P6R1T2V5W7",
    issuer="end-user-7f3k",
    feedback_id="answer_quality",
    value="up",
    revision=1,
    idempotency_key="idem_01J9Z2K5Q7X3M8N4P6R1T2V5W8",
    occurred_at=datetime(2026, 9, 21, 12, 0, 0, tzinfo=timezone.utc),
)
print(receipt.accepted, receipt.duplicate, receipt.current, receipt.seq)
# True False up 18446744073709551615

const receipt = await gateway.submitFeedback({
  logicalRequestId: "lrq_01J9Z2K5Q7X3M8N4P6R1T2V5W7",
  issuer: "end-user-7f3k",
  feedbackId: "answer_quality",
  value: "up",
  revision: 1,
  idempotencyKey: "idem_01J9Z2K5Q7X3M8N4P6R1T2V5W8",
  occurredAt: new Date("2026-09-21T12:00:00.000Z"),
});

Each field carries one decision:

A rating with no neutral is a choice you inherit. An unrated request is absent from the population rather than sitting in the middle of it, which is what lets the coverage figures below mean anything.

Retry without double-counting

The receipt is the ledger’s own statement. seq is a 64-bit counter carried as a decimal string, because the largest value it can hold does not survive a double, and match: pending says the join to the request happens at query time rather than now.

Send the identical rating a second time and it comes back as a success that changed nothing:

accepted: False duplicate: True
current: up revision: 1 stored_revisions: 1
seq: 18446744073709551615 str match: pending

That script printed the type of seq beside it: str, because the client keeps the counter as a string and parses it into no number of its own.

Let somebody change their mind

A correction is the next revision of the same (logical_request_id, issuer, feedback_id), not a second rating. One issuer may revise up to five times inside a seven-day window from the request, and the gateway admits 60 ratings a minute per service.

The highest stored revision wins on read, whatever order the revisions arrived in. A revision that arrives late is a slow network and does not overwrite a newer verdict, and current on each receipt tells you which value now wins.

Two shapes are refused before anything leaves your process, which keeps a typo out of your retry budget:

class: ConfigurationError code: feedback_value_invalid
message: value must be "up" or "down". There is no neutral rating: an unrated request is absent from the population, which is a different fact from a middle score.

class: ConfigurationError code: feedback_revision_invalid
message: revision counts from 1 per (request, issuer, feedback_id) and must be a positive integer. A correction is the NEXT revision.

A sixth revision is not one of them: the client sends it and the gateway answers 412, because whether the revisions are spent is the server’s fact and not the client’s.

Tell the six refusals apart

Each status is a different answer, and collapsing any pair would tell a caller to retry something permanent or to abandon an outage. The Python classes are below; TypeScript raises the same names.

The 429 is the only one carrying a wait, and the client reports what the gateway sent rather than inventing a backoff:

class: RateLimitError
code: feedback_rate_limited
status: 429
retry_after_seconds: 1
message: Too many ratings from this runtime key. Retry shortly.

Read how much traffic anybody rated

The gateway Overview carries a feedback coverage card, one block per activity purpose, because a production request and a playground experiment are two populations.

Coverage divides distinct rated requests (app_feedback_rated_requests.v1) by eligible requests (requests.v1). Both use requests admitted in the selected window. Two people rating one of ten requests means one rated request and 10% coverage, not 20%. Replayed feedback and revisions cannot increase that count.

A valid rating may arrive after the selected window closes. It counts toward the window in which its request was admitted, up to the read’s evaluation time and within the seven-day feedback limit. A rating for a request admitted before the window does not increase this window’s coverage. Refresh to include newly indexed feedback.

Ratings received in this window is a separate count, app_feedback.v1. It counts latest verdict identities (logical_request_id, issuer, feedback_id) by feedback time, so it can exceed the number of rated requests. It is never used as the coverage numerator. The card keeps both counts and the eligible population visible.

When no request in the cohort has feedback, the card says No feedback yet. Missing measurements stay unavailable. If eligible requests are only a floor because capture is incomplete, the card shows the counts without a share: dividing by that floor would overstate coverage.

The card also splits ratings by where they came from and never merges the sources: an application’s users, a member of staff reviewing an answer, and a judge model scoring one are three different claims. On this build only the app-reported row has a producer, so it is the only row that renders. The positive and negative halves render as named unavailable reasons for the same cause: nothing publishes that split yet, and inferring it from a count would invent the one number a team would act on.

Each group offers a drill-down button, “Show eligible requests”, which opens the request list for that purpose over the same window. The list is the group’s eligible requests; this build publishes no per-request verdict to filter it by, so read it as the population the figures were taken over.

Read one request’s feedback

The coverage card counts rated requests. When somebody complains about one answer, you want the verdicts on that one request: who rated it, which way, at which revision, and whether the rating has joined the request yet.

This is a management-lane read, not a gateway call. It needs a management key holding metadata.read on the scope. The ttv_ runtime key that submits ratings cannot make it, and feedback.write grants no read. The issuer’s own label also needs content.read, as the issuer handle explains.

A verdict is windowed on when it was given, its occurred_at, in a half-open UTC window [from, to). To read everything one request can ever receive, start the window at the request’s admitted_at from its receipt and end it at the earlier of now, rounded up to the minute, and admitted_at plus seven days and one minute.

The portal does not show a request’s app feedback yet. A feedback section on the request receipt is planned for a later release; until it arrives, use the CLI or an SDK.

From the CLI

--grain feedback with --request scopes the page to one request on the server:

Not yet executed against a deployment.

$ tokentriage cloud insights drilldown --grain feedback \
    --request lrq_K11111111111 \
    --org org_K11111111111 --workspace t_K11111111111 \
    --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z
cloud: customer.insights.query POST http://127.0.0.1:18701/portal/api/v1/customer/v1/organizations/org_K11111111111/workspaces/t_K11111111111/insights/query
lrq_K11111111111	fis_fedcba9876543210fedcba9876543210	fb-7c1e	up	rev 1	pending	app-reported
lrq_K11111111111	fis_0123456789abcdef0123456789abcdef	fb-2a90	down	rev 3	matched	app-reported
lrq_K11111111111	unassigned	fb-0001	up	rev 1	matched	app-reported
cloud: some verdicts were stored before issuer handles existed and show issuer `unassigned`; rows of one issuer may not group together
cloud: gateway_journal is behind (produced 918290, indexed 918273), so the most recent traffic may be missing

Each line is one verdict: request, issuer, feedback_id, value, revision, match state, and app-reported. The cloud: lines went to stderr. That run used a CLI built from source against a loopback server replaying the recorded case drilldown_feedback_request, so the 127.0.0.1 address is that server’s, passed with --base-url and left out of the command line shown; you see your own base URL. The watermark line comes from the recorded read context, whose gateway journal had not finished indexing.

A key that also holds content.read gets the issuer’s label in the second column, replayed from drilldown_feedback_content:

Not yet executed against a deployment.

cloud: customer.insights.query POST http://127.0.0.1:18702/portal/api/v1/customer/v1/organizations/org_K11111111111/workspaces/t_K11111111111/insights/query
lrq_K11111111111	svc.search-ranker	fb-7c1e	up	rev 1	pending	app-reported
lrq_K11111111111	user-4821	fb-2a90	down	rev 3	matched	app-reported
lrq_K11111111111	legacy:user-7	fb-0001	up	rev 1	matched	app-reported
cloud: some verdicts were stored before issuer handles existed and show issuer `unassigned`; rows of one issuer may not group together
cloud: gateway_journal is behind (produced 918290, indexed 918273), so the most recent traffic may be missing

--match reads one match state across the window instead of one request. This replays drilldown_feedback_unmatched:

Not yet executed against a deployment.

$ tokentriage cloud insights drilldown --grain feedback --match unmatched \
    --org org_K11111111111 --workspace t_K11111111111 \
    --from 2026-09-20T12:00:00Z --to 2026-09-21T12:00:00Z
cloud: customer.insights.query POST http://127.0.0.1:18703/portal/api/v1/customer/v1/organizations/org_K11111111111/workspaces/t_K11111111111/insights/query
lrq_P11111111111	fis_0123456789abcdef0123456789abcdef	fb-9d02	down	rev 1	unmatched	app-reported
lrq_Q11111111111	fis_fedcba9876543210fedcba9876543210	fb-4410	up	rev 2	unmatched	app-reported
cloud: gateway_journal is behind (produced 918290, indexed 918273), so the most recent traffic may be missing

tokentriage cloud request get --feedback reads the receipt, then makes the feedback read as a second call over that request’s admission window, at the request’s own purpose, and prints it under App feedback. Against a deployment older than this change, which refuses the grain, replayed from drilldown_feedback_invalid_filter after a recorded receipt, it prints:

Not yet executed against a deployment.

$ tokentriage cloud request get --org org_ABCDEFGHJKLM --workspace t_ABCDEFGHJKLM \
    --request lreq_2026_09_20_000001 --feedback
cloud: customer.requests.get GET http://127.0.0.1:18705/portal/api/v1/customer/v1/organizations/org_ABCDEFGHJKLM/workspaces/t_ABCDEFGHJKLM/requests/lreq_2026_09_20_000001
request   lreq_2026_09_20_000001
status    succeeded
served by gateway gw_ABCDEFGHJKLM, environment env_ABCDEFGHJKLM, release rel_ABCDEFGHJKLM
purpose   application (chat, streaming)
acted as  runtime_credential A1 production key
attempt   att_2026_09_20_000001 primary via openai/gpt-4o-mini
usage     812 in, 240 out, 128 cached tokens (provider_reported)
cost      unpriced (no price has been settled for this attempt yet)
guards    clean
note: this request has no cost figure (unpriced (no price has been settled for this attempt yet)). It is NOT $0.00.

App feedback
cloud: customer.insights.query POST http://127.0.0.1:18705/portal/api/v1/customer/v1/organizations/org_ABCDEFGHJKLM/workspaces/t_ABCDEFGHJKLM/insights/query
  not served by this deployment: it refused the feedback grain (HTTP 400 invalid_query)
Error: app feedback not served by this deployment: it refused the feedback grain (HTTP 400 invalid_query): customer.insights.query was refused (HTTP 400); detail: insightcontract: invalid document; type=invalid_query; request_id=req_drilldown_feedback_invalid_filter

The receipt lines come from a recorded receipt, and both addresses are the replay server’s. The receipt above the section stands. The command still exits 1, so a script that asked for the feedback knows it did not get it. Each refused read names its own cause:

The status and slug in parentheses are the server’s own.

From Python and TypeScript

Both SDKs send the read through insights.query with drilldown: "feedback" and a feedback_filter. The examples sdk/python/examples/observability_request_diagnosis.py and sdk/ts/examples/observability_request_diagnosis.ts make it after the receipt read, with the window computed from admitted_at by feedback_window and feedbackWindow.

Not yet executed against a deployment.

import os
from tokentriage.customer import (CustomerClient, InsightForbiddenError,
                                  InsightRefusalError, InsightUnavailableError)

client = CustomerClient(os.environ["TOKENTRIAGE_BASE_URL"],
                        os.environ["TOKENTRIAGE_MANAGEMENT_KEY"])
try:
    answer = client.insights.query("org_K11111111111", "t_K11111111111", {
        "schema": 1, "shape": "drilldown", "drilldown": "feedback", "page_size": 50,
        "from": "2026-09-20T12:00:00Z", "to": "2026-09-21T12:00:00Z",
        "purposes": ["application"],
        "feedback_filter": {"logical_request_id": "lrq_K11111111111"},
    })["data"]
except InsightForbiddenError as error:  # 403: a permission, not an outage
    print("denied", error.status, error.code)
except InsightUnavailableError as error:  # 503: retry later
    print("unavailable", error.status, error.code, error.next_action)
except InsightRefusalError as error:  # 400/422: this deployment does not serve the grain
    print("not served", error.status, error.code, error.next_action)
else:
    for row in answer.get("feedback", []):
        print(row["issuer_ref"], row.get("issuer", "-"), row["feedback_id"],
              row["value"], "rev", row["revision"], row["match"],
              row.get("purpose", "-"))
    print("coverage:", answer.get("coverage", []))

import {
  CustomerClient, InsightForbiddenError, InsightRefusalError,
  InsightUnavailableError, type InsightAnswer,
} from "tokentriage/customer";

const client = new CustomerClient({
  baseUrl: process.env.TOKENTRIAGE_BASE_URL!,
  managementApiKey: process.env.TOKENTRIAGE_MANAGEMENT_KEY!,
});
try {
  const answer = (await client.insights.query("org_K11111111111", "t_K11111111111", {
    schema: 1, shape: "drilldown", drilldown: "feedback", page_size: 50,
    from: "2026-09-20T12:00:00Z", to: "2026-09-21T12:00:00Z",
    purposes: ["application"],
    feedback_filter: { logical_request_id: "lrq_K11111111111" },
  })).data as InsightAnswer;
  for (const row of answer.feedback ?? []) {
    console.log(row.issuer_ref, row.issuer ?? "-", row.feedback_id, row.value,
      "rev", row.revision, row.match, row.purpose ?? "-");
  }
  console.log("coverage:", answer.coverage ?? []);
} catch (error) {
  if (error instanceof InsightForbiddenError) {
    console.log("denied", error.status, error.code);         // 403: a permission
  } else if (error instanceof InsightUnavailableError) {
    console.log("unavailable", error.status, error.code, error.nextAction); // 503
  } else if (error instanceof InsightRefusalError) {
    console.log("not served", error.status, error.code, error.nextAction);  // 400/422
  } else {
    throw error;
  }
}

The Python program ran with the in-repo SDK (sdk/python/src) under Python 3.14.5, three times, against a loopback server replaying drilldown_feedback_request, drilldown_feedback_content and drilldown_feedback_invalid_filter in turn. It printed:

fis_fedcba9876543210fedcba9876543210 - fb-7c1e up rev 1 pending -
fis_0123456789abcdef0123456789abcdef - fb-2a90 down rev 3 matched application
unassigned - fb-0001 up rev 1 matched application
coverage: ['issuer_ref_unassigned']

fis_fedcba9876543210fedcba9876543210 svc.search-ranker fb-7c1e up rev 1 pending -
fis_0123456789abcdef0123456789abcdef user-4821 fb-2a90 down rev 3 matched application
unassigned legacy:user-7 fb-0001 up rev 1 matched application
coverage: ['issuer_ref_unassigned']

not served 400 invalid_query check_selection

The TypeScript program has not been run. Its calls mirror sdk/ts/test/unit/customer_insights.test.ts, which replays the same three cases.

Read the three match states

Every row carries match, the state of its join to the request it names:

pending is “not yet”, and unmatched is “no such request”, not “deleted”. While the gateway journal’s progress is unknown, a row stays pending rather than turning unmatched. A rating is filed in the scope of the key that submitted it, so the recorded pending row above, filed under another application, joins no request there.

Newest revision wins

A row is one verdict identity (request, issuer, feedback_id) at its newest revision, whatever order the revisions arrived in. revision is your application’s own number, carried as a decimal string, and value and occurred_at are that revision’s. A row with a revision above 1 was revised. There is no count of earlier revisions: storage may already have folded them together, and a count would overstate the trail.

Reading changes nothing about writing. A pending row is not a reason to resend a rating: the retry rules above still apply, and resending the identical document with its original idempotency key comes back as a duplicate that changed nothing.

Tell issuers apart

issuer_ref is on every row: an opaque handle, fis_ followed by 32 lowercase hex characters, stored the first time the workspace sees an issuer and not derived from the label. Key your rows on it. A verdict stored before handles existed shows unassigned, and the answer then carries the coverage token issuer_ref_unassigned once; rows of one such issuer may not group together.

issuer, the label your application sent, is added only when the reader also holds content.read, on the same rows and the same handles. The CLI prints the label when the server sent one and the handle otherwise.

App-reported, and which purpose a row belongs to

Every row carries source: app_reported, which the CLI prints as app-reported: your application said so, and the gateway did not observe it.

A matched row carries its request’s purpose and is filtered by the purposes the read asked for. A pending or unmatched row has joined no request, so its purpose is unknown: it is included whatever purposes you asked for, with purpose absent. Do not count those rows toward a purpose.

What this lane does not carry

There is no tokentriage cloud feedback command, and the customer-API client has no feedback operation in either SDK: a management key holding feedback.write would be a wide credential doing a narrow credential’s job. No deployment runs the feedback grain yet, so the aggregate counts on the card are the only served read surface in production; the registered read is above. A rating carries no free-text note, and nothing on this path reads a rating back out through the gateway.

A score your own system produces about a request, such as a resolved support ticket, is not a rating. Report outcomes about your requests describes the reported-outcome operations for that claim.