# How TokenTriage works Follow one model request from your app to its provider, then see how it becomes a costed ledger record. Source: https://tokentriage.com/docs/concepts/how-it-works --- import Callout from '../../../components/Callout.astro'; import NextSteps from '../../../components/NextSteps.astro'; import RequestFlowDiagram from '../../../components/RequestFlowDiagram.astro'; Your provider invoice tells you the total amount spent. It does not connect that total to the individual requests that caused it: a large prompt, a retry, an agent step, or a particular model. TokenTriage makes that connection by sitting between your app and its model provider. It forwards each request and writes one record with its usage, price source, and cost. This page follows that default path. You will see what your app changes, what the provider returns, and what TokenTriage records afterward. Routing, caching, governance, and guardrails are optional; they appear near the end. You do not need TokenTriage running to read this page. To run it without a provider key, configuration file, or account, use the [Quickstart](/docs/get-started/quickstart). ## What your app changes Your app still sends a normal provider request. It points its model client's base URL at TokenTriage instead of calling the provider directly. For example, with TokenTriage running and configured for OpenAI, a chat request can go to the local TokenTriage address: ```console $ curl -i http://127.0.0.1:8787/v1/chat/completions \ -H "Content-Type: application/json" \ -H "Authorization: Bearer $OPENAI_API_KEY" \ -d '{"model":"gpt-5","messages":[{"role":"user","content":"Hello"}]}' HTTP/1.1 200 OK content-type: application/json x-tt-request-id: tt_d5k7ntwm6utqbk4eigg3am3udm ... ``` That command makes a request to your provider account and may incur provider charges. The response above is abbreviated, and the request ID is illustrative. Use the Quickstart if you want a local mock request with no provider charge. For this example, the base URL is the only application change. The path, authorization header, model name, and JSON body remain OpenAI-shaped. TokenTriage also accepts Anthropic-shaped requests at `/v1/messages`. The docs call these two API formats **dialects**. With the default features enabled, TokenTriage forwards the provider's response body without translating it into another API format. It adds the `x-tt-request-id` response header shown above. That ID connects the response to the ledger record written for the request. Cost is not returned as a response header. TokenTriage records it after the provider has reported usage, which may arrive with the final chunk of a streamed response. If the provider does not report usage, TokenTriage can estimate it and labels the resulting figure as an estimate. ## Follow the default request path The request has a short path through the system: Read the top row from left to right, then follow the return arrow along the bottom. The ledger and admin API sit beside that request path. 1. **Your app sends a provider-shaped request.** The default TokenTriage address is `127.0.0.1:8787`, which is reachable only from the same machine unless you change the bind. 2. **TokenTriage forwards it to the configured provider.** Your provider account, credentials, and bill remain with the provider. A new installation records the selected model without changing it. 3. **The provider's response returns to your app.** TokenTriage passes the response body through in the same provider format and adds the request ID header. 4. **TokenTriage appends one ledger row.** Once usage is known, it records the model, token counts, pricing result, and request ID in `~/.tokentriage/decisions.jsonl`. The docs call port `8787` the **data plane** because it carries application traffic. A second listener on port `9090`, the **admin plane**, reads the ledger for health, metrics, and dashboard views. Provider traffic never uses the admin port. [Architecture](/docs/concepts/architecture) explains the two listeners and deployment options. An accounting error does not block a request that TokenTriage can otherwise forward. TokenTriage logs the error and continues with the provider response. The gateway is still a network hop: if the process or its network path is unavailable, requests cannot pass through it. ## Read the ledger record The ledger is a JSONL file: one JSON object per line. You can inspect the most recent row with any tool that reads text: ```console $ tail -n 1 ~/.tokentriage/decisions.jsonl ``` Here is the example row, pretty-printed and trimmed to the fields used on this page: ```json { "req_id": "tt_d5k7ntwm6utqbk4eigg3am3udm", "http": { "path": "/v1/chat/completions", "dialect": "openai", "stream": false, "status": 200 }, "request": { "model": "gpt-5", "prompt_sha256": "b1e4…", "prompt_bytes": 5, "messages": 1 }, "decision": { "mode": "ledger", "tier": "", "upstream": "local" }, "usage": { "input_tokens": 88, "output_tokens": 40, "cache_read_tokens": 32, "total_tokens": 160, "source": "provider" }, "cost": { "usd": 0.000514, "pricing_key": "openai/gpt-5", "pricing_snapshot": "litellm@2026-07-14#179affb", "estimated": false, "would_usd": null } } ``` Start with three parts of the row: - **`req_id` identifies the same request as the response header.** You can use it to move from an application response to the corresponding ledger row. - **`usage` records the token counts.** Its `source` field says whether the provider reported those counts or TokenTriage estimated them. - **`cost` records the pricing result.** It includes the dollar amount, the model's pricing key, the version of the price table used, and whether the figure rests on estimated usage. The ledger does not store the prompt or response text. The `request` object contains a fingerprint and counted features such as byte and message counts. See [Ledger fields](/docs/reference/ledger-fields) for the complete record. ## See where the cost comes from The example cost is arithmetic you can check: ```text 88 × $1.25e-6 + 40 × $1e-5 + 32 × $1.25e-7 = $0.000514 ``` Those are the input, output, and cache-read token counts multiplied by the corresponding prices for `openai/gpt-5`. TokenTriage reads those prices from a table compiled into the binary, so calculating the cost does not require a network call. The `pricing_snapshot` identifies the dated version of that table. Keeping it on the row means you can tell which prices were used even after the binary is updated. Two cases need different labels: - If usage is estimated, the CLI and dashboard mark the figure `EST`. - If the snapshot has no price for the model, they show `UNPRICED` instead of `$0.00`. That distinction keeps unknown spend out of priced totals without making it disappear. [The honesty model](/docs/concepts/honesty-model) explains the pricing and evidence rules in detail. ## Nothing was routed, and that's the default The example row has `"mode": "ledger"`. A new installation starts in this mode: it forwards the requested model and records what happened. It does not select a different model. `shadow` mode can record what a routing policy would have selected without changing the request. `route` mode can act on that decision. [Core concepts](/docs/concepts/core-concepts#the-trust-ladder) compares the three modes. ## What changes when you enable more features The default path stays small. Other features alter a specific part of it only after you configure them: | Feature | What it can change | |---|---| | [Routing](/docs/build/routing) | Selects a configured model instead of always using the requested one. | | [Response caching](/docs/build/caching) | May return a stored response after the routing decision. | | [Governance](/docs/build/governance) | Can stop a request before it reaches the provider when a configured limit is exceeded. | | [Guardrails](/docs/guardrails/overview) | Can inspect, block, or change configured input and output content. | These features are separate from recording usage and cost. You can use the ledger without enabling any of them. ## Limits and failure behavior Three boundaries are worth keeping in mind: - **TokenTriage is part of the live network path.** Internal accounting failures fall back to forwarding, but a stopped process or unreachable host cannot forward a request. - **The admin API is separate from provider traffic.** If port `9090` is unavailable, the request listener can still run, but the dashboard and metrics endpoint are unavailable. - **A missing cost is not a zero.** Unpriced requests remain visible as an excluded group. Estimated requests remain visible with their estimate label. This page covers the default request and its ledger row. It does not cover deployment layouts, the complete ledger schema, or how a routing policy makes a decision.