Report outcomes about your requests

Your helpdesk knows which answers closed a ticket. Your test harness knows which generated query ran. The gateway saw the same requests and knows only what they cost and how long they took.

A reported outcome carries that knowledge back. Your system sends a small scalar claim about one request it made through a gateway: a value, a pass, or both, for a named metric. The service stores it with its provenance and, later, matches it to the request the gateway recorded.

This page covers the hosted customer API through HTTP, either SDK and the tokentriage cloud CLI. If a person rated an answer inside your application, you want app feedback instead; the next section says which is which.

Feedback, reported outcomes and Quality evaluations

Three features attach a judgement to a request, and each one answers a different question. Picking the wrong one mixes claims that should stay apart.

A reported outcome is not a TokenTriage evaluation. It is stored as your claim, labelled with the class of credential that sent it, and kept apart from feedback. For the evaluations TokenTriage runs itself, see Compare cost and quality before you switch models.

What you need

You do not need the gateway’s runtime key or content.read. No outcome carries prompt or completion text.

Write one outcome

An outcome is one JSON object. This one says a support ticket raised from request lreq_2026_09_20_000001 was resolved:

{
  "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01",
  "request_id": "lreq_2026_09_20_000001",
  "application_id": "app_ABCDEFGHJKLM",
  "gateway_id": "gw_ABCDEFGHJKLM",
  "environment_id": "env_ABCDEFGHJKLM",
  "metric_id": "support_resolution",
  "metric_version": "1",
  "evaluator_id": "ticket-resolution-webhook",
  "evaluator_version": "webhook-v1",
  "source": "external_reported",
  "value": 1,
  "pass": true,
  "revision": "1",
  "occurred_at": "2026-09-20T12:04:00Z",
  "attribution": "helpdesk:ticket-48213"
}

The IDs are synthetic. support_resolution and ticket-resolution-webhook are a metric and an evaluator from the built-in registry. Each field has one job:

The server fills in who sent the outcome, its basis and its arrival time. A body naming an actor is turned away with HTTP 422. attribution is kept by the admission ledger and is not returned by a read.

Submit a batch

One submission carries 1 to 50 items in at most 65,536 bytes of JSON. Each observation_id may appear once per submission. The answer names every item, so one bad item does not sink the others.

Over HTTP, wrap the items in {"schema": 1, "items": [...]} and post them:

Not yet executed against a deployment.

$ curl -sS -X POST \
    "$TOKENTRIAGE_BASE_URL/portal/api/v1/customer/v1/organizations/org_ABCDEFGHJKLM/workspaces/t_ABCDEFGHJKLM/observations" \
    -H "Authorization: Bearer $TOKENTRIAGE_MANAGEMENT_KEY" \
    -H 'content-type: application/json' \
    --data-binary @submission.json

An HTTP client stays under both limits itself. The SDKs and the CLI take any number of items and split them, in order, into requests within 50 items and 65,536 bytes each, measured on the bytes they send. They send the requests one after another and return one disposition per item in input order. An item too large for any request is turned away before anything is sent.

From either SDK, with the item above saved in outcomes.json as a one-element array:

import json
import os
from tokentriage.customer import CustomerClient, UnsupportedError

client = CustomerClient(os.environ["TOKENTRIAGE_BASE_URL"],
                        os.environ["TOKENTRIAGE_MANAGEMENT_KEY"])
with open("outcomes.json", encoding="utf-8") as handle:
    items = json.load(handle)
try:
    receipt = client.observations.submit("org_ABCDEFGHJKLM", "t_ABCDEFGHJKLM",
                                         items)
except UnsupportedError as error:  # HTTP 501: no reported-outcome admission here
    print(type(error).__name__, error.status)
    print(error)
else:
    for d in receipt["items"]:
        print(d["observation_id"], d["state"], d.get("reason", "-"))
    print(receipt["match"], "over", receipt["requests"], "request(s)")

import { readFileSync } from "node:fs";
import { CustomerClient, UnsupportedError } from "tokentriage/customer";

const client = new CustomerClient({
  baseUrl: process.env.TOKENTRIAGE_BASE_URL!,
  managementApiKey: process.env.TOKENTRIAGE_MANAGEMENT_KEY!,
});
const items = JSON.parse(readFileSync("outcomes.json", "utf8"));
try {
  const receipt = await client.observations.submit(
    "org_ABCDEFGHJKLM", "t_ABCDEFGHJKLM", items);
  for (const d of receipt.items) {
    console.log(d.observation_id, d.state, d.reason ?? "-");
  }
  console.log(receipt.match, "over", receipt.requests, "request(s)");
} catch (error) {
  // UnsupportedError: HTTP 501, no reported-outcome admission here.
  if (!(error instanceof UnsupportedError)) throw error;
  console.log(error.name, error.status);
  console.log(error.message);
}

Neither program has been run against a deployment. A control plane with no reported-outcome admission answers HTTP 501, which both clients raise as UnsupportedError.

From the CLI, --file takes either a bare array of items or a whole submission document:

Not yet executed against a deployment; output omitted.

$ tokentriage cloud observation submit --file outcomes.json \
    --org org_ABCDEFGHJKLM --workspace t_ABCDEFGHJKLM \
    --base-url https://app.example.com

The next section shows what it prints.

Read a partial batch

The receipt holds one disposition per item and a match that is always pending: the request is matched later, when a read asks. An item is accepted (stored now), duplicate (stored earlier; nothing changed), conflict or invalid. The last two carry a reason, and neither is stored.

This receipt answers four items. The first is new, the second was sent before, the third reuses a stored revision with a different value, and the fourth names metric_version "v1":

Not yet executed against a deployment.

{
  "schema": "customer/v1",
  "request_id": "req_01J9Z2K5Q7X3M8N4P6R1T2V5W9",
  "kind": "outcome-receipt",
  "data": {
    "schema": 1,
    "items": [
      { "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01", "state": "accepted" },
      { "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f02", "state": "duplicate" },
      { "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f03", "state": "conflict",
        "reason": "revision_conflict" },
      { "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f04", "state": "invalid",
        "reason": "unknown_vocabulary" }
    ],
    "match": "pending"
  }
}

The HTTP answer wraps the receipt in the customer API’s envelope; the SDKs return the receipt itself, with requests added. The CLI prints a table and a tally, then exits 3 because two items were not stored:

Not yet executed against a deployment.

$ tokentriage cloud observation submit --file outcomes.json \
    --org org_ABCDEFGHJKLM --workspace t_ABCDEFGHJKLM \
    --base-url https://app.example.com
OBSERVATION                           STATE      REASON
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01  accepted   -
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f02  duplicate  -
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f03  conflict   revision_conflict
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f04  invalid    unknown_vocabulary
1 accepted, 1 duplicate, 1 conflict, 1 invalid
4 items in 1 request(s); match is pending until the request is indexed

That table was rendered with the CLI’s own format strings over this synthetic receipt. On stderr the CLI adds one line:

cloud: 2 item(s) were answered conflict or invalid and were not stored; the other items stand

Each per-item reason names one fix:

Correct an outcome

A correction is the next revision of the same observation_id, not a new outcome. Revision 1 fixes the request, the place, the metric and evaluator with their versions, the attribution and occurred_at. A later revision may change only value and pass, up to revision 5.

Send each revision in its own submission, because one submission names each observation_id once. Revisions may arrive in any order. A read reports the highest stored revision as current, so a late revision 2 does not overwrite a revision 3 that arrived first.

Identity is per sender. Two credentials reporting the same observation_id create two outcomes, told apart on read by their actor_ref.

Read outcomes back

A read lists the outcomes whose occurred_at falls in a half-open UTC window of at most 400 days, optionally for one request. By default it returns each outcome’s current revision; revisions returns every stored revision with its current flag. Each row carries a match state:

Over HTTP, post the read to observations/read:

Not yet executed against a deployment.

$ curl -sS -X POST \
    "$TOKENTRIAGE_BASE_URL/portal/api/v1/customer/v1/organizations/org_ABCDEFGHJKLM/workspaces/t_ABCDEFGHJKLM/observations/read" \
    -H "Authorization: Bearer $TOKENTRIAGE_MANAGEMENT_KEY" \
    -H 'content-type: application/json' \
    -d '{"schema":1,"from":"2026-09-20T00:00:00Z","to":"2026-09-21T00:00:00Z"}'

The answer’s data holds read_context, rows, truncated, and next_cursor when truncated is true. To continue, send the same read with cursor set to next_cursor and read_context copied unchanged. A cursor without its read context is rejected. One row, as the contract defines it:

Not yet executed against a deployment.

{
  "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01",
  "request_id": "lreq_2026_09_20_000001",
  "application_id": "app_ABCDEFGHJKLM",
  "gateway_id": "gw_ABCDEFGHJKLM",
  "environment_id": "env_ABCDEFGHJKLM",
  "metric_id": "support_resolution",
  "metric_version": "1",
  "evaluator_id": "ticket-resolution-webhook",
  "evaluator_version": "webhook-v1",
  "source": "external_reported",
  "basis": "reported",
  "actor_class": "service",
  "actor_ref": "roa_0123456789abcdef0123456789abcdef",
  "value": 1,
  "pass": true,
  "revision": "1",
  "current": true,
  "occurred_at": "2026-09-20T12:04:00Z",
  "arrived_at": "2026-09-20T12:04:02Z",
  "match": "matched"
}

basis, actor_class, actor_ref and arrived_at are the server’s. actor_class is person or service. actor_ref is an opaque stored reference to the sender, not derived from its identity, and attribution is absent.

The SDKs follow next_cursor the same way. This loop is the one in the SDK examples (sdk/python/examples/reported_outcomes.py, sdk/ts/examples/reported_outcomes.ts):

Not yet executed against a deployment.

def read_all(client, org, workspace, query, max_pages=10):
    """Every row of the window, following next_cursor with its read_context."""
    page = client.observations.list(org, workspace, query)
    rows = list(page["rows"])
    pages = 1
    while page.get("truncated") and page.get("next_cursor") and pages < max_pages:
        page = client.observations.list(org, workspace, query,
                                        cursor=page["next_cursor"],
                                        read_context=page["read_context"])
        rows.extend(page["rows"])
        pages += 1
    return rows

rows = read_all(client, "org_ABCDEFGHJKLM", "t_ABCDEFGHJKLM",
                {"schema": 1, "from": "2026-09-20T00:00:00Z",
                 "to": "2026-09-21T00:00:00Z"})
for row in rows:
    measure = row.get("value", row.get("pass"))
    print(f"{row['request_id']} {row['metric_id']}={measure} "
          f"rev {row['revision']} "
          f"({row['basis']} by {row['actor_class']}, {row['match']})")

async function readAll(
  client: CustomerClient,
  org: string,
  workspace: string,
  query: { schema: number; from: string; to: string },
  maxPages = 10,
) {
  let page = await client.observations.list(org, workspace, query);
  const rows = [...page.rows];
  for (let pages = 1; page.truncated && page.next_cursor && pages < maxPages; pages++) {
    page = await client.observations.list(org, workspace, query, {
      cursor: page.next_cursor,
      readContext: page.read_context,
    });
    rows.push(...page.rows);
  }
  return rows;
}

const rows = await readAll(client, "org_ABCDEFGHJKLM", "t_ABCDEFGHJKLM", {
  schema: 1,
  from: "2026-09-20T00:00:00Z",
  to: "2026-09-21T00:00:00Z",
});
for (const row of rows) {
  console.log(
    `${row.request_id} ${row.metric_id}=${row.value ?? row.pass} ` +
      `rev ${row.revision} (${row.basis} by ${row.actor_class}, ${row.match})`,
  );
}

For the row above, each program would print lreq_2026_09_20_000001 support_resolution=1 rev 1 (reported by service, matched).

The CLI follows every continuation itself. --request narrows to one request and --revisions lists every stored revision:

Not yet executed against a deployment.

$ tokentriage cloud observation list \
    --from 2026-09-20T00:00:00Z --to 2026-09-21T00:00:00Z \
    --org org_ABCDEFGHJKLM --workspace t_ABCDEFGHJKLM \
    --base-url https://app.example.com
OBSERVATION                           REQUEST                 REV  CURRENT  METRIC                EVALUATOR                             VALUE  PASS   ACTOR    OCCURRED              MATCH
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01  lreq_2026_09_20_000001  1    true     support_resolution@1  ticket-resolution-webhook@webhook-v1  1      true   service  2026-09-20T12:04:00Z  matched
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f02  lreq_2026_09_20_000002  1    true     support_resolution@1  ticket-resolution-webhook@webhook-v1  0      false  service  2026-09-20T12:09:00Z  pending
Not shown: the request index has not caught up, so a pending match may still become matched (request_index_behind)
2 row(s) across 1 page(s)

That table was rendered with the CLI’s format strings over two synthetic rows. The Not shown: line is the answer’s coverage: request_index_behind says a pending row may still match, and outcomes_not_projected says outcomes admitted recently may not be visible yet.

What can go wrong

A refused submission stores nothing from that request. The statuses are distinct because their remedies are: one says fix the body, one says ask for access, one says wait.

Neither SDK retries a 429 or a 503 on its own, and the CLI does not either: it prints the Retry-After it received. The two Observations… classes subclass InsightRefusalError and InsightUnavailableError, so a handler for those catches them too. Both are importable from tokentriage.customer (Python) and tokentriage/customer (TypeScript).

Reads have their own set. HTTP 403 means metadata.read does not reach the workspace. HTTP 422 means the read is malformed: the window, a page_size outside 1 to 500, a bad request ID, or a cursor without its read context. HTTP 410 means the read context expired, so start again without cursor and read_context. HTTP 503 means the read could not be answered. Read request coverage and freshness covers read contexts and those answers in depth.

What this does not carry

An outcome is a scalar claim with its provenance. It has no free text, no notes, no rationale and no cost field, so none can be sent. It does not change any request count, cost or feedback coverage figure, and it is not scored or re-graded by TokenTriage. The metrics and evaluators it can name are the registered ones; this contract has no way to add your own.