# Report outcomes about your requests

Send your own systems' scores about gateway requests, read them back with their match state, and tell them apart from feedback and Quality evaluations.

Source: https://tokentriage.com/docs/build/reported-outcomes

---
import Callout from '../../../components/Callout.astro';
import CodeTabs from '../../../components/CodeTabs.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';

<Callout type="note">

**Registered, not yet captured from a deployment.** This page describes the registered
operations `customer.observations.submit` and `customer.observations.list`, which both SDKs
and the CLI send. Every block marked "Not yet executed against a deployment" shows the
contract, not a captured answer.

</Callout>

Your helpdesk knows which answers closed a ticket. Your test harness knows which generated
query ran. The gateway saw the same requests and knows only what they cost and how long they
took.

A reported outcome carries that knowledge back. Your system sends a small scalar claim about
one request it made through a gateway: a value, a pass, or both, for a named metric. The
service stores it with its provenance and, later, matches it to the request the gateway
recorded.

This page covers the hosted customer API through HTTP, either SDK and the `tokentriage cloud`
CLI. If a person rated an answer inside your application, you want
[app feedback](/docs/build/report-app-feedback) instead; the next section says which is which.

## Feedback, reported outcomes and Quality evaluations

Three features attach a judgement to a request, and each one answers a different question.
Picking the wrong one mixes claims that should stay apart.

| | App feedback | Reported outcome | Quality evaluation |
|---|---|---|---|
| who judges | a person using your application | your own system: a helpdesk, a test harness, a batch job | TokenTriage, running a scorer you declared |
| what it says | `up` or `down` for one question about one request | a value, a pass, or both, for a registered metric and evaluator | graded results for saved test cases against two configurations |
| credential | a `ttv_` runtime key holding `feedback.write`, at the gateway | a management key whose access includes `observations.write`, at the customer API | a management key, at the customer API |
| basis | a person's rating | always `reported`: a claim the service relays and did not observe | what the scorer measured |
| read back as | coverage figures, with no per-rating rows | one row per observation, with its match state | the comparison's evidence |

A reported outcome is not a TokenTriage evaluation. It is stored as your claim, labelled with
the class of credential that sent it, and kept apart from feedback. For the evaluations
TokenTriage runs itself, see
[Compare cost and quality before you switch models](/docs/build/compare-cost-and-quality).

## What you need

| You need | Because |
|---|---|
| your organization ID (`org_…`) and workspace ID (`t_…`) | outcomes are submitted and read inside one workspace |
| a management key in `TOKENTRIAGE_MANAGEMENT_KEY` | it identifies who reported; the body never names an actor |
| `observations.write` at the place each outcome names | checked per item, against its application, gateway and environment |
| `metadata.read` on the workspace | to read outcomes back |
| the request's ID, as the gateway returned it in `x-tt-request-id` | an outcome is about one logical request |
| the application, gateway and environment IDs that served it | each outcome names its place, and the service checks it |

You do not need the gateway's runtime key or `content.read`. No outcome carries prompt or
completion text.

## Write one outcome

An outcome is one JSON object. This one says a support ticket raised from request
`lreq_2026_09_20_000001` was resolved:

```json
{
  "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01",
  "request_id": "lreq_2026_09_20_000001",
  "application_id": "app_ABCDEFGHJKLM",
  "gateway_id": "gw_ABCDEFGHJKLM",
  "environment_id": "env_ABCDEFGHJKLM",
  "metric_id": "support_resolution",
  "metric_version": "1",
  "evaluator_id": "ticket-resolution-webhook",
  "evaluator_version": "webhook-v1",
  "source": "external_reported",
  "value": 1,
  "pass": true,
  "revision": "1",
  "occurred_at": "2026-09-20T12:04:00Z",
  "attribution": "helpdesk:ticket-48213"
}
```

The IDs are synthetic. `support_resolution` and `ticket-resolution-webhook` are a metric and
an evaluator from the built-in registry. Each field has one job:

| Field | What it decides |
|---|---|
| `observation_id` | the outcome's identity: a lowercase UUID you mint once and reuse for every revision |
| `request_id` | the request it is about: letters, digits, `_` and `-`, at most 128 |
| `application_id`, `gateway_id`, `environment_id` | the place that served the request; all three are required |
| `metric_id`, `evaluator_id` | a registered metric, and a registered evaluator that declares it |
| `metric_version` | `"1"` for every metric on this contract |
| `evaluator_version` | the evaluator's registered version, such as `webhook-v1` |
| `source` | always `external_reported` |
| `value` | finite; `0` or `1` for a binary metric, `0` to `1` for a score, a whole number of `0` or more for a count |
| `pass` | allowed only on a metric that declares a pass floor; send `value`, `pass` or both |
| `revision` | `"1"` for the first report, up to `"5"`, as a string |
| `occurred_at` | when the outcome happened, in UTC |
| `attribution` | optional label for your own correlation: letters, digits and `.`, `_`, `:`, `-`, at most 128 |

The server fills in who sent the outcome, its basis and its arrival time. A body naming an
`actor` is turned away with HTTP 422. `attribution` is kept by the admission ledger and is
not returned by a read.

## Submit a batch

One submission carries 1 to 50 items in at most 65,536 bytes of JSON. Each
`observation_id` may appear once per submission. The answer names every item, so one bad item
does not sink the others.

Over HTTP, wrap the items in `{"schema": 1, "items": [...]}` and post them:

_Not yet executed against a deployment._

```console
$ curl -sS -X POST \
    "$TOKENTRIAGE_BASE_URL/portal/api/v1/customer/v1/organizations/org_ABCDEFGHJKLM/workspaces/t_ABCDEFGHJKLM/observations" \
    -H "Authorization: Bearer $TOKENTRIAGE_MANAGEMENT_KEY" \
    -H 'content-type: application/json' \
    --data-binary @submission.json
```

An HTTP client stays under both limits itself. The SDKs and the CLI take any number of items
and split them, in order, into requests within 50 items and 65,536 bytes each, measured on
the bytes they send. They send the requests one after another and return one disposition per
item in input order. An item too large for any request is turned away before anything is
sent.

From either SDK, with the item above saved in `outcomes.json` as a one-element array:

<CodeTabs>
  <Fragment slot="python">
  ```python
  import json
  import os
  from tokentriage.customer import CustomerClient, UnsupportedError

  client = CustomerClient(os.environ["TOKENTRIAGE_BASE_URL"],
                          os.environ["TOKENTRIAGE_MANAGEMENT_KEY"])
  with open("outcomes.json", encoding="utf-8") as handle:
      items = json.load(handle)
  try:
      receipt = client.observations.submit("org_ABCDEFGHJKLM", "t_ABCDEFGHJKLM",
                                           items)
  except UnsupportedError as error:  # HTTP 501: no reported-outcome admission here
      print(type(error).__name__, error.status)
      print(error)
  else:
      for d in receipt["items"]:
          print(d["observation_id"], d["state"], d.get("reason", "-"))
      print(receipt["match"], "over", receipt["requests"], "request(s)")
  ```
  </Fragment>
  <Fragment slot="typescript">
  ```typescript
  import { readFileSync } from "node:fs";
  import { CustomerClient, UnsupportedError } from "tokentriage/customer";

  const client = new CustomerClient({
    baseUrl: process.env.TOKENTRIAGE_BASE_URL!,
    managementApiKey: process.env.TOKENTRIAGE_MANAGEMENT_KEY!,
  });
  const items = JSON.parse(readFileSync("outcomes.json", "utf8"));
  try {
    const receipt = await client.observations.submit(
      "org_ABCDEFGHJKLM", "t_ABCDEFGHJKLM", items);
    for (const d of receipt.items) {
      console.log(d.observation_id, d.state, d.reason ?? "-");
    }
    console.log(receipt.match, "over", receipt.requests, "request(s)");
  } catch (error) {
    // UnsupportedError: HTTP 501, no reported-outcome admission here.
    if (!(error instanceof UnsupportedError)) throw error;
    console.log(error.name, error.status);
    console.log(error.message);
  }
  ```
  </Fragment>
</CodeTabs>

Neither program has been run against a deployment. A control plane with no reported-outcome
admission answers HTTP 501, which both clients raise as `UnsupportedError`.

From the CLI, `--file` takes either a bare array of items or a whole submission document:

_Not yet executed against a deployment; output omitted._

```console
$ tokentriage cloud observation submit --file outcomes.json \
    --org org_ABCDEFGHJKLM --workspace t_ABCDEFGHJKLM \
    --base-url https://app.example.com
```

The next section shows what it prints.

## Read a partial batch

The receipt holds one disposition per item and a `match` that is always `pending`: the
request is matched later, when a read asks. An item is `accepted` (stored now), `duplicate`
(stored earlier; nothing changed), `conflict` or `invalid`. The last two carry a `reason`,
and neither is stored.

This receipt answers four items. The first is new, the second was sent before, the third
reuses a stored revision with a different value, and the fourth names `metric_version`
`"v1"`:

_Not yet executed against a deployment._

```json
{
  "schema": "customer/v1",
  "request_id": "req_01J9Z2K5Q7X3M8N4P6R1T2V5W9",
  "kind": "outcome-receipt",
  "data": {
    "schema": 1,
    "items": [
      { "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01", "state": "accepted" },
      { "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f02", "state": "duplicate" },
      { "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f03", "state": "conflict",
        "reason": "revision_conflict" },
      { "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f04", "state": "invalid",
        "reason": "unknown_vocabulary" }
    ],
    "match": "pending"
  }
}
```

The HTTP answer wraps the receipt in the customer API's envelope; the SDKs return the receipt
itself, with `requests` added. The CLI prints a table and a tally, then exits 3 because two
items were not stored:

_Not yet executed against a deployment._

```console
$ tokentriage cloud observation submit --file outcomes.json \
    --org org_ABCDEFGHJKLM --workspace t_ABCDEFGHJKLM \
    --base-url https://app.example.com
OBSERVATION                           STATE      REASON
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01  accepted   -
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f02  duplicate  -
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f03  conflict   revision_conflict
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f04  invalid    unknown_vocabulary
1 accepted, 1 duplicate, 1 conflict, 1 invalid
4 items in 1 request(s); match is pending until the request is indexed
```

That table was rendered with the CLI's own format strings over this synthetic receipt. On
stderr the CLI adds one line:

```text
cloud: 2 item(s) were answered conflict or invalid and were not stored; the other items stand
```

| CLI exit | Means |
|---|---|
| `0` | every item is `accepted` or `duplicate` |
| `3` | some item is `conflict` or `invalid` |
| `1` | a request failed; the items answered before it are still printed |

<Callout type="key">

Resubmitting a whole batch is safe: nothing already stored is changed. A stored item answers
`duplicate` only while its `occurred_at` is still inside the 7-day window and your access to
its place is unchanged; otherwise the retry answers `invalid` (`outside_window` or
`scope_not_authorized`) although the item is stored. So after a timeout, a 503 or a stopped
run, send the same items again promptly rather than working out which ones landed.

</Callout>

Each per-item reason names one fix:

| `reason` | State | What it means | What to do |
|---|---|---|---|
| `unknown_vocabulary` | `invalid` | the metric or evaluator is not registered, the evaluator does not declare the metric, a version is not the registered one, `metric_version` is not `"1"`, or `source` is not `external_reported` | correct the names or versions |
| `invalid_value` | `invalid` | a field is missing or malformed, the value is outside the metric's range, `pass` was sent for a metric with no pass floor, or neither `value` nor `pass` was sent | correct the item |
| `oversize` | `invalid` | a field is longer than 128 bytes | shorten it |
| `outside_window` | `invalid` | `occurred_at` is outside the admission window | report closer to when it happened; the window is the last seven days |
| `scope_not_authorized` | `invalid` | the application, gateway and environment are not a place your access can report on | check the three IDs, or ask for `observations.write` there |
| `revision_conflict` | `conflict` | this revision is stored with a different body | send the change as the next revision |
| `attribution_mismatch` | `conflict` | a later revision changes a field the first revision fixed | keep those fields as revision 1 sent them |

## Correct an outcome

A correction is the next revision of the same `observation_id`, not a new outcome. Revision
1 fixes the request, the place, the metric and evaluator with their versions, the
attribution and `occurred_at`. A later revision may change only `value` and `pass`, up to
revision 5.

Send each revision in its own submission, because one submission names each
`observation_id` once. Revisions may arrive in any order. A read reports the highest stored
revision as `current`, so a late revision 2 does not overwrite a revision 3 that arrived
first.

Identity is per sender. Two credentials reporting the same `observation_id` create two
outcomes, told apart on read by their `actor_ref`.

## Read outcomes back

A read lists the outcomes whose `occurred_at` falls in a half-open UTC window of at most 400
days, optionally for one request. By default it returns each outcome's current revision;
`revisions` returns every stored revision with its `current` flag. Each row carries a match
state:

| `match` | Means |
|---|---|
| `matched` | the request was found in the same application, gateway and environment |
| `pending` | the request index has not caught up yet, so this may still become `matched` |
| `unmatched` | no such request was found in that place |

Over HTTP, post the read to `observations/read`:

_Not yet executed against a deployment._

```console
$ curl -sS -X POST \
    "$TOKENTRIAGE_BASE_URL/portal/api/v1/customer/v1/organizations/org_ABCDEFGHJKLM/workspaces/t_ABCDEFGHJKLM/observations/read" \
    -H "Authorization: Bearer $TOKENTRIAGE_MANAGEMENT_KEY" \
    -H 'content-type: application/json' \
    -d '{"schema":1,"from":"2026-09-20T00:00:00Z","to":"2026-09-21T00:00:00Z"}'
```

The answer's `data` holds `read_context`, `rows`, `truncated`, and `next_cursor` when
`truncated` is true. To continue, send the same read with `cursor` set to `next_cursor` and
`read_context` copied unchanged. A cursor without its read context is rejected. One row, as
the contract defines it:

_Not yet executed against a deployment._

```json
{
  "observation_id": "3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01",
  "request_id": "lreq_2026_09_20_000001",
  "application_id": "app_ABCDEFGHJKLM",
  "gateway_id": "gw_ABCDEFGHJKLM",
  "environment_id": "env_ABCDEFGHJKLM",
  "metric_id": "support_resolution",
  "metric_version": "1",
  "evaluator_id": "ticket-resolution-webhook",
  "evaluator_version": "webhook-v1",
  "source": "external_reported",
  "basis": "reported",
  "actor_class": "service",
  "actor_ref": "roa_0123456789abcdef0123456789abcdef",
  "value": 1,
  "pass": true,
  "revision": "1",
  "current": true,
  "occurred_at": "2026-09-20T12:04:00Z",
  "arrived_at": "2026-09-20T12:04:02Z",
  "match": "matched"
}
```

`basis`, `actor_class`, `actor_ref` and `arrived_at` are the server's. `actor_class` is
`person` or `service`. `actor_ref` is an opaque stored reference to the sender, not derived
from its identity, and `attribution` is absent.

The SDKs follow `next_cursor` the same way. This loop is the one in the SDK examples
(`sdk/python/examples/reported_outcomes.py`, `sdk/ts/examples/reported_outcomes.ts`):

_Not yet executed against a deployment._

<CodeTabs>
  <Fragment slot="python">
  ```python
  def read_all(client, org, workspace, query, max_pages=10):
      """Every row of the window, following next_cursor with its read_context."""
      page = client.observations.list(org, workspace, query)
      rows = list(page["rows"])
      pages = 1
      while page.get("truncated") and page.get("next_cursor") and pages < max_pages:
          page = client.observations.list(org, workspace, query,
                                          cursor=page["next_cursor"],
                                          read_context=page["read_context"])
          rows.extend(page["rows"])
          pages += 1
      return rows

  rows = read_all(client, "org_ABCDEFGHJKLM", "t_ABCDEFGHJKLM",
                  {"schema": 1, "from": "2026-09-20T00:00:00Z",
                   "to": "2026-09-21T00:00:00Z"})
  for row in rows:
      measure = row.get("value", row.get("pass"))
      print(f"{row['request_id']} {row['metric_id']}={measure} "
            f"rev {row['revision']} "
            f"({row['basis']} by {row['actor_class']}, {row['match']})")
  ```
  </Fragment>
  <Fragment slot="typescript">
  ```typescript
  async function readAll(
    client: CustomerClient,
    org: string,
    workspace: string,
    query: { schema: number; from: string; to: string },
    maxPages = 10,
  ) {
    let page = await client.observations.list(org, workspace, query);
    const rows = [...page.rows];
    for (let pages = 1; page.truncated && page.next_cursor && pages < maxPages; pages++) {
      page = await client.observations.list(org, workspace, query, {
        cursor: page.next_cursor,
        readContext: page.read_context,
      });
      rows.push(...page.rows);
    }
    return rows;
  }

  const rows = await readAll(client, "org_ABCDEFGHJKLM", "t_ABCDEFGHJKLM", {
    schema: 1,
    from: "2026-09-20T00:00:00Z",
    to: "2026-09-21T00:00:00Z",
  });
  for (const row of rows) {
    console.log(
      `${row.request_id} ${row.metric_id}=${row.value ?? row.pass} ` +
        `rev ${row.revision} (${row.basis} by ${row.actor_class}, ${row.match})`,
    );
  }
  ```
  </Fragment>
</CodeTabs>

For the row above, each program would print
`lreq_2026_09_20_000001 support_resolution=1 rev 1 (reported by service, matched)`.

The CLI follows every continuation itself. `--request` narrows to one request and
`--revisions` lists every stored revision:

_Not yet executed against a deployment._

```console
$ tokentriage cloud observation list \
    --from 2026-09-20T00:00:00Z --to 2026-09-21T00:00:00Z \
    --org org_ABCDEFGHJKLM --workspace t_ABCDEFGHJKLM \
    --base-url https://app.example.com
OBSERVATION                           REQUEST                 REV  CURRENT  METRIC                EVALUATOR                             VALUE  PASS   ACTOR    OCCURRED              MATCH
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f01  lreq_2026_09_20_000001  1    true     support_resolution@1  ticket-resolution-webhook@webhook-v1  1      true   service  2026-09-20T12:04:00Z  matched
3f1c6a2e-8b4d-4c7a-9e21-5d0b7a6c1f02  lreq_2026_09_20_000002  1    true     support_resolution@1  ticket-resolution-webhook@webhook-v1  0      false  service  2026-09-20T12:09:00Z  pending
Not shown: the request index has not caught up, so a pending match may still become matched (request_index_behind)
2 row(s) across 1 page(s)
```

That table was rendered with the CLI's format strings over two synthetic rows. The
`Not shown:` line is the answer's `coverage`: `request_index_behind` says a `pending` row may still
match, and `outcomes_not_projected` says outcomes admitted recently may not be visible yet.

## What can go wrong

A refused submission stores nothing from that request. The statuses are distinct because
their remedies are: one says fix the body, one says ask for access, one says wait.

| Status | `type` | When | Python and TypeScript class | What to do |
|---|---|---|---|---|
| 422 | `validation_failed` | an unknown member (such as `actor`), a `NaN` or `1e999` token, a body over 65,536 bytes, `schema` other than 1, 0 or more than 50 items, or a malformed or repeated `observation_id` | `InsightRefusalError` | fix the body and send it again |
| 403 | `forbidden` | no `observations.write` anywhere in the workspace, or a `system` credential | `InsightForbiddenError` | ask a workspace administrator for a binding that grants it |
| 403 | `authority_revoked` | your access changed while the batch was admitted | `InsightRefusalError` | resubmit; it is judged under your current access |
| 404 | `not_found` | the workspace is not one your key can see: "this organization has no such workspace" | `CustomerProblemError` | check the organization and workspace IDs |
| 429 | `rate_limited` | too many outcomes from this sender in this workspace; `Retry-After` names the wait | `ObservationsRateLimitedError` | wait `retry_after` (Python) or `retryAfterSeconds` (TypeScript), then resubmit |
| 503 | `dependency_unavailable` | the store did not answer, or writes are switched off on this deployment | `ObservationsUnavailableError` | retry after `Retry-After` when present; resubmitting is safe |
| 501 | `not_implemented` | this control plane has no reported-outcome admission | `UnsupportedError` | use a deployment that serves the operation |

Neither SDK retries a 429 or a 503 on its own, and the CLI does not either: it prints the
`Retry-After` it received. The two `Observations…` classes subclass `InsightRefusalError`
and `InsightUnavailableError`, so a handler for those catches them too. Both are importable
from `tokentriage.customer` (Python) and `tokentriage/customer` (TypeScript).

Reads have their own set. HTTP 403 means `metadata.read` does not reach the workspace. HTTP
422 means the read is malformed: the window, a `page_size` outside 1 to 500, a bad request ID,
or a cursor without its read context. HTTP 410 means the read context expired, so start again
without `cursor` and `read_context`. HTTP 503 means the read could not be answered.
[Read request coverage and freshness](/docs/operate/request-coverage) covers read contexts
and those answers in depth.

<Details summary="Where the statuses and messages on this page come from">

The statuses, `type` values and sentences come from
`internal/control/foundation/httpapi/observations.go`, the contract
`internal/central/insightcontract/outcome_submission.go`, the vocabulary check
`internal/evaluator/reported.go`, the SDK clients and `internal/cli/cloud_observations.go`
at this page's date. None of them has been answered by a deployment.

</Details>

## What this does not carry

An outcome is a scalar claim with its provenance. It has no free text, no notes, no
rationale and no cost field, so none can be sent. It does not change any request count, cost
or feedback coverage figure, and it is not scored or re-graded by TokenTriage. The metrics and
evaluators it can name are the registered ones; this contract has no way to add your own.

<NextSteps items={[
  { title: "Report and read app feedback", href: "/docs/build/report-app-feedback", description: "When a person using your application rates an answer, rather than your system reporting a score." },
  { title: "Compare cost and quality before you switch models", href: "/docs/build/compare-cost-and-quality", description: "The evaluations TokenTriage runs itself, with a scorer you declare and a bar set in advance." },
  { title: "Trace one request from its ID", href: "/docs/build/observability-request-id", description: "Find the request an outcome is about, and the place that served it." },
  { title: "Read request coverage and freshness", href: "/docs/operate/request-coverage", description: "Why a request may not be indexed yet, which is what a pending match is waiting for." }
]} />