How TokenTriage works

Your provider invoice tells you the total amount spent. It does not connect that total to the individual requests that caused it: a large prompt, a retry, an agent step, or a particular model.

TokenTriage makes that connection by sitting between your app and its model provider. It forwards each request and writes one record with its usage, price source, and cost.

This page follows that default path. You will see what your app changes, what the provider returns, and what TokenTriage records afterward. Routing, caching, governance, and guardrails are optional; they appear near the end.

You do not need TokenTriage running to read this page. To run it without a provider key, configuration file, or account, use the Quickstart.

What your app changes

Your app still sends a normal provider request. It points its model client’s base URL at TokenTriage instead of calling the provider directly.

For example, with TokenTriage running and configured for OpenAI, a chat request can go to the local TokenTriage address:

$ curl -i http://127.0.0.1:8787/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer $OPENAI_API_KEY" \
  -d '{"model":"gpt-5","messages":[{"role":"user","content":"Hello"}]}'
HTTP/1.1 200 OK
content-type: application/json
x-tt-request-id: tt_d5k7ntwm6utqbk4eigg3am3udm
...

That command makes a request to your provider account and may incur provider charges. The response above is abbreviated, and the request ID is illustrative. Use the Quickstart if you want a local mock request with no provider charge.

For this example, the base URL is the only application change. The path, authorization header, model name, and JSON body remain OpenAI-shaped. TokenTriage also accepts Anthropic-shaped requests at /v1/messages. The docs call these two API formats dialects.

With the default features enabled, TokenTriage forwards the provider’s response body without translating it into another API format. It adds the x-tt-request-id response header shown above. That ID connects the response to the ledger record written for the request.

Cost is not returned as a response header. TokenTriage records it after the provider has reported usage, which may arrive with the final chunk of a streamed response. If the provider does not report usage, TokenTriage can estimate it and labels the resulting figure as an estimate.

Follow the default request path

The request has a short path through the system:

Read the top row from left to right, then follow the return arrow along the bottom. The ledger and admin API sit beside that request path.

1. Your app sends a provider-shaped request. The default TokenTriage address is 127.0.0.1:8787, which is reachable only from the same machine unless you change the bind.
2. TokenTriage forwards it to the configured provider. Your provider account, credentials, and bill remain with the provider. A new installation records the selected model without changing it.
3. The provider’s response returns to your app. TokenTriage passes the response body through in the same provider format and adds the request ID header.
4. TokenTriage appends one ledger row. Once usage is known, it records the model, token counts, pricing result, and request ID in ~/.tokentriage/decisions.jsonl.

The docs call port 8787 the data plane because it carries application traffic. A second listener on port 9090, the admin plane, reads the ledger for health, metrics, and dashboard views. Provider traffic never uses the admin port. Architecture explains the two listeners and deployment options.

Read the ledger record

The ledger is a JSONL file: one JSON object per line. You can inspect the most recent row with any tool that reads text:

$ tail -n 1 ~/.tokentriage/decisions.jsonl

Here is the example row, pretty-printed and trimmed to the fields used on this page:

{
  "req_id": "tt_d5k7ntwm6utqbk4eigg3am3udm",
  "http": { "path": "/v1/chat/completions", "dialect": "openai", "stream": false, "status": 200 },
  "request": { "model": "gpt-5", "prompt_sha256": "b1e4…", "prompt_bytes": 5, "messages": 1 },
  "decision": { "mode": "ledger", "tier": "", "upstream": "local" },
  "usage": { "input_tokens": 88, "output_tokens": 40, "cache_read_tokens": 32, "total_tokens": 160, "source": "provider" },
  "cost": { "usd": 0.000514, "pricing_key": "openai/gpt-5", "pricing_snapshot": "litellm@2026-07-14#179affb", "estimated": false, "would_usd": null }
}

Start with three parts of the row:

• req_id identifies the same request as the response header. You can use it to move from an application response to the corresponding ledger row.
• usage records the token counts. Its source field says whether the provider reported those counts or TokenTriage estimated them.
• cost records the pricing result. It includes the dollar amount, the model’s pricing key, the version of the price table used, and whether the figure rests on estimated usage.

The ledger does not store the prompt or response text. The request object contains a fingerprint and counted features such as byte and message counts. See Ledger fields for the complete record.

See where the cost comes from

The example cost is arithmetic you can check:

88 × $1.25e-6 + 40 × $1e-5 + 32 × $1.25e-7 = $0.000514

Those are the input, output, and cache-read token counts multiplied by the corresponding prices for openai/gpt-5. TokenTriage reads those prices from a table compiled into the binary, so calculating the cost does not require a network call.

The pricing_snapshot identifies the dated version of that table. Keeping it on the row means you can tell which prices were used even after the binary is updated.

Two cases need different labels:

• If usage is estimated, the CLI and dashboard mark the figure EST.
• If the snapshot has no price for the model, they show UNPRICED instead of $0.00.

That distinction keeps unknown spend out of priced totals without making it disappear. The honesty model explains the pricing and evidence rules in detail.

Nothing was routed, and that’s the default

The example row has "mode": "ledger". A new installation starts in this mode: it forwards the requested model and records what happened. It does not select a different model.

shadow mode can record what a routing policy would have selected without changing the request. route mode can act on that decision. Core concepts compares the three modes.

What changes when you enable more features

The default path stays small. Other features alter a specific part of it only after you configure them:

Feature	What it can change
Routing	Selects a configured model instead of always using the requested one.
Response caching	May return a stored response after the routing decision.
Governance	Can stop a request before it reaches the provider when a configured limit is exceeded.
Guardrails	Can inspect, block, or change configured input and output content.

These features are separate from recording usage and cost. You can use the ledger without enabling any of them.

Limits and failure behavior

Three boundaries are worth keeping in mind:

• TokenTriage is part of the live network path. Internal accounting failures fall back to forwarding, but a stopped process or unreachable host cannot forward a request.
• The admin API is separate from provider traffic. If port 9090 is unavailable, the request listener can still run, but the dashboard and metrics endpoint are unavailable.
• A missing cost is not a zero. Unpriced requests remain visible as an excluded group. Estimated requests remain visible with their estimate label.

This page covers the default request and its ledger row. It does not cover deployment layouts, the complete ledger schema, or how a routing policy makes a decision.