# Outcomes and feedback

Send quality signals back against a request id, and see which fields the SDK withholds because they are the server's to decide.

Source: https://tokentriage.com/docs/sdks/outcomes

---
import Callout from '../../../components/Callout.astro';
import NextSteps from '../../../components/NextSteps.astro';
import Details from '../../../components/Details.astro';
import CodeTabs from '../../../components/CodeTabs.astro';

The ledger knows what every request cost. It does not know whether any of them were any good.
Two write paths close that gap, and both join on the request id you already have.

`ledger.feedback` carries one thumbs-up or thumbs-down. `ledger.outcomes` carries one or more
scored metric observations. Both append companion rows; neither edits the original.

This page covers how to send them and, more usefully, which of the fields you might try to send
are refused. That second part is where the surface has opinions.

## A thumbs verdict

The smallest possible signal, and the one to reach for when a human clicked something.

<CodeTabs>
  <Fragment slot="python">
  ```python
  from tokentriage import ledger

  ack = ledger.feedback(rcpt, ok=False, note="hallucinated the version number")
  print(ack.accepted, ack.verdict, ack.source, ack.actor, ack.ts)
  ```
  </Fragment>
  <Fragment slot="typescript">
  ```ts
  const ack = await tt.ledger.feedback(rcpt, {
    ok: false,
    note: "hallucinated the version number",
  });
  console.log(ack.accepted, ack.verdict, ack.source, ack.actor, ack.ts);
  ```
  </Fragment>
</CodeTabs>

```text
True down human local 2026-08-10T13:55:50.798Z
```

You supplied the verdict and the note. `source`, `actor` and `ts` came back from the server; they
are provenance, and a client that could set them could manufacture evidence about who said what
and when.

The body is a fixed three-key object, with `note` omitted entirely when empty rather than sent as
an empty string, and no cost field at all. Feedback carries labels, not money.

<Callout type="warning">
`feedback()` is the one ledger call that is not shielded, in either language. It raises on
failure rather than degrading, because a silently dropped write is worse than a loud one: you
would believe you had recorded a judgement that no row holds.
</Callout>

## Scored observations

`ledger.outcomes` submits one or more metric observations against a request. Each becomes an
append-only companion row.

<CodeTabs>
  <Fragment slot="python">
  ```python
  ack = ledger.outcomes(
      rcpt,
      [
          {"metric": "answer_relevance", "value": 0.82},
          {"metric": "task_success", "value": 1, "pass": True},
      ],
      source="human_review",
      evaluator="analyst_panel",
      eval_ver="v1",
  )
  ```
  </Fragment>
  <Fragment slot="typescript">
  ```ts
  const ack = await tt.ledger.outcomes(rcpt, {
    metrics: [
      { metric: "answer_relevance", value: 0.82 },
      { metric: "task_success", value: 1, pass: true },
    ],
    source: "human_review",
    evaluator: "analyst_panel",
    evalVer: "v1",
  });
  ```
  </Fragment>
</CodeTabs>

Four things are required and none has a default. `metric`, `evaluator` and `source` must be ids
your server has registered. `eval_ver` is the evaluator's version, and it is required because a
swapped scorer is a version boundary: an observation that cannot say which version produced it
cannot be excluded when that version is retired.

Each metric needs a numeric `value`. `pass` is optional and must be a genuine boolean if present,
because an absent pass verdict is not a false one.

Delayed submission is the normal case. An annotation or a business signal may arrive weeks after
the request, and the server does not require the request id to still be around.

### Per-item refusals, not exceptions

A bad row does not discard the batch. Unregistered ids come back as per-item refusals with a code
and a pointer at the offending input:

```text
accepted: 0 refused: 2
  [0] accepted=False basis=None actor=None
     refusal: evaluator_unregistered | /outcomes/0/evaluator | not a registered evaluator id (§2.3)
  [1] accepted=False basis=None actor=None
     refusal: metric_unregistered | /outcomes/1/metric | not a registered metric id (§2.3).
              An outcome naming a metric nobody declared is a permanent ledger line no surface
              can interpret
```

That capture is from a test harness with no metric registry configured, which is why both items
were turned away. On a configured server the accepted items come back with `accepted=True` and a
server-assigned `basis`.

Check `ack.accepted` and `ack.refused` rather than assuming a 200 meant everything landed. Both
are absent rather than zero when the server did not say, since an absent `refused` read as zero
is the one thing an ingestion client must not assume.

## What the SDK withholds

This is the part worth reading even if you never call `outcomes`.

Five fields are accepted by the API and **never sent**. If you supply one, the key is withheld
from the request and the acknowledgement tells you so, with a stable code and a JSON pointer at
the position that would have carried it:

```text
  IGNORED client_basis_ignored at /outcomes/0/basis
  IGNORED client_basis_ignored at /outcomes/1/basis
```

| Field | Why it is not yours to state |
|---|---|
| `basis` | the server computes it from registries a client cannot see |
| `actor` | derived from the authenticated principal; a client-set one manufactures evidence |
| `ts` | the server's arrival stamp; a client-set one could backdate into a published window |
| `judge` | claims about a validation run neither the SDK nor the ingestion API performed |
| `eval_cost` | money is server-produced; the SDK does not compute a dollar |

Any key the wire contract does not carry is withheld the same way, under
`client_field_ignored`, rather than dropped.

<Callout type="key">
The point of disclosing rather than stripping is that a silently discarded field is
indistinguishable from an honoured one at the call site. You would believe you had recorded a
measurement basis, and nothing would ever contradict you. Read `ack.ignored` if you passed
anything beyond `metric`, `value` and `pass`.
</Callout>

`basis` is the interesting one. It describes what kind of evidence an observation is, and the
stored value is the minimum of several registered ceilings — the metric's ground-truth class, the
evaluator's kind, the source's declared ceiling. A client can see none of those registries, so a
client-asserted basis is a claim about a measurement it did not make. The value the server settled
on comes back in `ack.results[i].basis`.

<Details summary="Where the boundary with Calibration and Evals sits">

These two calls are the input side of evaluation, and this section documents them as write paths
only: the call shapes, what the SDK sends, what it withholds, the per-item refusals, and the
credential protocol they share with the ledger read.

What an outcome *means* belongs to Calibration & Evals: the metric registry, what makes an id
registered, the basis vocabulary and the ceiling rule that produces it, evaluator kinds, and what
consumes these rows once written. This page names `basis` and says the server decides it; it does
not teach the algorithm.

The practical test for a future author: if a sentence would still be true with a different client
sending the same HTTP request, it belongs in Calibration & Evals rather than here.

</Details>

## The credential protocol

Both writes run the same containment protocol as the ledger read: plane-collision refusal before
any socket, probe before auth, no redirects, and the token from `TOKENTRIAGE_ADMIN_TOKEN` only.
A misdirected POST that carried your admin token to a provider is precisely the hazard that
protocol exists to stop, and a write is the case where it matters most.

[Fetching a costed row](/docs/sdks/ledger#the-credential-protocol) has the four rules and what
probe-before-auth does not prove.

<Details summary="Why the request body is assembled by hand">

Both bindings build the JSON body themselves rather than handing an object to a serialiser, and
they reimplement JavaScript's number rendering to do it.

`json.dumps(1.0)` is `1.0` in Python and `1` in JavaScript, and Python's `1e-07` is JavaScript's
`1e-7`. The obvious implementation therefore puts the two bindings on different bytes for the same
observation. Label keys are sorted for the same reason, since map iteration order is a different
fact in each language.

A non-finite value is rejected rather than sent. `JSON.stringify(NaN)` produces `null`, which
would land a null where a number belongs and read downstream as "no observation" rather than "a
broken one". Seven of the 70 conformance vectors pin these bytes, and five more pin the feedback
body.

</Details>

<NextSteps items={[
  { title: 'Fetching a costed row', href: '/docs/sdks/ledger', description: 'The read path, and the credential protocol these writes share.' },
  { title: 'The management client', href: '/docs/sdks/management', description: 'The generated client for the rest of the admin API.' },
  { title: 'Calibration and evals', href: '/docs/calibration/overview', description: 'What these rows feed, and what a basis actually means.' },
  { title: 'The contract and how it evolves', href: '/docs/sdks/contract', description: 'What the conformance corpus pins, including these bodies.' }
]} />