# Core concepts

Understand what TokenTriage records for each request and how its three modes move from observation to active routing.

Source: https://tokentriage.com/docs/concepts/core-concepts

---
import Callout from '../../../components/Callout.astro';
import NextSteps from '../../../components/NextSteps.astro';
import TrustLadderDiagram from '../../../components/TrustLadderDiagram.astro';
import DecisionLedgerDiagram from '../../../components/DecisionLedgerDiagram.astro';

Your provider invoice tells you the total amount due. It doesn't tell you which requests caused
the cost, which model handled them, or what changed from last month. TokenTriage fills in that
missing detail by recording each request as it passes through the proxy.

When you're ready to reduce that cost, TokenTriage can also choose among models and providers you
configure. It starts by observing traffic without changing it. You decide when it may compare
other routes and when it may apply them.

Two ideas explain that progression: the **decision ledger** records what happened, and the
**trust ladder** controls what TokenTriage is allowed to change. If LLM gateways are new to you,
start with [LLM gateways, proxies, and routers](/docs/introduction/llm-gateways-explained). For a
guided tour of one request, read [How TokenTriage works](/docs/concepts/how-it-works).

## The decision ledger

Each proxied request appends one **decision row** to
`~/.tokentriage/decisions.jsonl`. JSONL is a plain-text format with one JSON object per line. You
can inspect it with ordinary text tools, and TokenTriage doesn't require a database.

The same file also records evaluations, guardrail actions, feedback, and oracle results. Those
events use their own row types, so not every line represents a request. Prompt text isn't stored
by default; a decision row stores a hash and measurements such as prompt size instead.

<DecisionLedgerDiagram />

Here is a shortened decision row:

```json
{
  "req_id": "tt_d5k7ntwm6utqbk4eigg3am3udm",
  "request": {
    "model": "gpt-5"
  },
  "decision": {
    "mode": "ledger",
    "upstream": "local"
  },
  "usage": {
    "input_tokens": 88,
    "output_tokens": 40,
    "source": "provider"
  },
  "cost": {
    "usd": 0.000514,
    "estimated": false
  }
}
```

From this row you can see which model was requested, how TokenTriage handled the request, how many
tokens the provider reported, and what the request cost. `usage.source: provider` means the token
counts came from the provider. `estimated: false` means the dollar figure is based on those
reported counts rather than a local estimate.

An **upstream** is a provider endpoint you configure. In this example, the request went to the
upstream named `local`. The complete row contains more detail for tracing, caching, pricing, and
routing; [Ledger fields](/docs/reference/ledger-fields) documents every field when you need it.

<Callout type="key">
**The CLI, dashboard, and exports read the same ledger.** Their totals agree because they use the
same rows. Derived data, such as the SQLite index used by the dashboard, can be rebuilt from this
file.
</Callout>

## The trust ladder

The next question is what TokenTriage may change about your traffic. The top-level `mode` setting
answers it:

```yaml
mode: ledger   # ledger | shadow | route
```

`ledger` is the default. A new installation records and costs requests but doesn't choose a
different model or provider. Each mode keeps the behavior of the mode before it and adds one
capability.

<TrustLadderDiagram />

| `mode` | What TokenTriage does | Can the selected model or provider change? |
|---|---|---|
| `ledger` | Records and costs the request, then sends it to the model your application requested | No |
| `shadow` | Also calculates which configured route it would choose and records the comparison | No |
| `route` | Applies the routing decision and sends the request to the selected target | Yes |

A routing **tier** is a named combination of one upstream and one model. In `shadow` mode,
TokenTriage compares tiers without changing the response your application receives. In `route`
mode, it may select a different tier, but your application still receives the same API format it
sent.

`eval` is a command, not a mode. Run it between `shadow` and `route` to check whether the less
expensive path still produces acceptable results. [Your first routing
rollout](/docs/get-started/first-routing-rollout) walks through the complete sequence.

## One proxy, two ports

TokenTriage keeps application traffic separate from its administrative tools. The docs call these
the **data plane** and the **admin plane**:

- The **data plane** listens on `127.0.0.1:8787` by default. Your application sends its OpenAI- or
  Anthropic-formatted requests here.
- The **admin plane** listens on `127.0.0.1:9090` by default. It serves health checks, metrics, API
  status, and the optional dashboard. It isn't part of the path a model request waits on.

Both ports listen only on the same machine by default. [Architecture](/docs/concepts/architecture)
shows the full topology and explains what to expose when you deploy TokenTriage elsewhere.

## When a cost can't be calculated

A successful request doesn't always produce a dollar figure. If the provider omits token counts,
TokenTriage estimates them when it can and labels the cost `EST`. If the model has no entry in the
price catalog, the cost is `UNPRICED`. Neither label means the request cost `$0.00`.

The decision row records which case occurred, so reports don't mix measured, estimated, and
missing costs. [Provenance and verdicts](/docs/concepts/honesty-model) explains how those labels
work across the rest of TokenTriage.

<NextSteps items={[
  { title: 'How TokenTriage works', href: '/docs/concepts/how-it-works', description: 'Follow one request from your application to the provider and back.' },
  { title: 'Your first routing rollout', href: '/docs/get-started/first-routing-rollout', description: 'Move from ledger to shadow, evaluate the result, and then enable routing.' },
  { title: 'Ledger fields', href: '/docs/reference/ledger-fields', description: 'Look up the complete schema when you need more than the shortened row shown here.' },
  { title: 'Glossary', href: '/docs/reference/glossary', description: 'Find definitions for the less common terms used throughout the documentation.' }
]} />