What is TokenTriage?

Your application calls a model API. At the end of the month, the provider sends a total. That total does not tell you which feature, customer, agent step, or retry caused the spending.

You could add the same accounting code to every application. TokenTriage gives those requests one shared place to pass through instead. Your application sends model requests to TokenTriage, which forwards them to your existing provider account and returns the provider’s response. At the same time, it writes one record per request to a local file called the decision ledger.

When TokenTriage has a price for the requested model, the ledger record includes the calculated cost. When it cannot price a model, it marks the record unpriced instead of reporting a misleading $0.

A new installation starts in ledger mode. It observes and records traffic; it does not switch models, return cached responses, inspect content, or enforce budgets until you enable those features.

This page assumes only that you have called a model API from code. If you want to see TokenTriage running first, the Quickstart uses built-in mock providers and requires no provider key, configuration file, or TokenTriage account.

Where it sits in your stack

A gateway is a shared stop between an application and another service. Model requests go to the gateway first; the gateway passes them to the provider. This lets you add accounting in one place instead of rebuilding it in every application.

Read the diagram from left to right. With direct calls, each application needs its own instrumentation if you want to connect provider spending to individual requests. With TokenTriage in the middle, the same traffic passes through one process that records each request as it happens.

Your provider accounts, credentials, and bills remain yours. TokenTriage is not a reseller or a hosted model service. The ledger write happens outside the request path, so your application does not wait for the accounting step. The gateway itself is still on the request path, so treat its availability as you would any other proxy. The deployment guide covers the available topologies.

What happens by default

A fresh installation has a narrow job: forward the request, return the response, and record its token use and cost.

• Your app changes one setting. Point its model API client’s base URL at TokenTriage instead of the provider.
• Your provider relationship stays the same. You use your own account and credentials, and continue paying the provider directly.
• Your client receives the provider’s response body. TokenTriage adds an x-tt-request-id response header that links the request to its ledger record.
• TokenTriage writes one local record. The default file is ~/.tokentriage/decisions.jsonl; each row carries the model, token counts, pricing status, and calculated cost when a price is available.

TokenTriage makes a few small changes to requests and headers so it can associate responses with ledger records and collect token counts from streaming calls. The HTTP headers reference documents the wire-level behavior. You do not need those details to start using the ledger.

In ledger mode, an internal processing error falls back to a direct provider pass-through and is written to the logs. That protects a provider response from an accounting failure. TokenTriage is still a network hop: if the process or its network path is unavailable, requests cannot pass through it.

What the ledger lets you answer

One request record gives you a model, token counts, a pricing result, and a request ID. Add labels, called dimensions, for a feature, tenant, or agent, and the same records can answer questions the provider invoice cannot:

• Which feature or tenant is responsible for the most spending?
• Did retries or unusually large prompts cause a cost spike?
• How does spending change across models or over time?
• Which group of requests is worth evaluating on a cheaper model?

Those optional dimensions do not change the provider request or response. They give you labels to group by on the command line, dashboard, and exports.

Routing, response caching, governance, guardrails, and evaluation are separate features. Each is off when you start and has its own configuration. Features at a glance shows what each feature does and where to read more.

Is TokenTriage a fit?

TokenTriage is a good fit when:

• several features, tenants, or agents share a provider account and you need to attribute their costs;
• you want evidence before moving some requests to a cheaper model; or
• you prefer to run the gateway yourself instead of sending prompts through another hosted service.

It is probably not the right fit when:

• you want a hosted URL and no infrastructure of your own;
• you need a provider or endpoint that TokenTriage does not currently cover — check Providers and dialects before adopting it; or
• you need software with years of production history. TokenTriage is beta.

What you need to run it

For your own traffic, you need:

• your existing provider account and credentials;
• an application or model client whose base URL you can configure; and
• somewhere to run the TokenTriage binary.

The default ledger setup needs no TokenTriage account, database server, or language runtime. You keep paying your providers as you do now. If you build from source instead of using a release binary, you also need Go.

This page stops at the basic mental model. The pages below cover the runnable first experience, the wider gateway vocabulary, optional capabilities, and product trade-offs.