# What is TokenTriage?

TokenTriage is a program you run between your app and its model providers to record usage and calculate per-request cost.

Source: https://tokentriage.com/docs/introduction/what-is-tokentriage

---
import GatewayPositionDiagram from '../../../components/GatewayPositionDiagram.astro';
import NextSteps from '../../../components/NextSteps.astro';

Your application calls a model API. At the end of the month, the provider sends a total. That
total does not tell you which feature, customer, agent step, or retry caused the spending.

You could add the same accounting code to every application. **TokenTriage gives those requests
one shared place to pass through instead.** Your application sends model requests to TokenTriage,
which forwards them to your existing provider account and returns the provider's response. At the
same time, it writes one record per request to a local file called the **decision ledger**.

When TokenTriage has a price for the requested model, the ledger record includes the calculated
cost. When it cannot price a model, it marks the record `unpriced` instead of reporting a
misleading `$0`.

A new installation starts in `ledger` mode. It observes and records traffic; it does not switch
models, return cached responses, inspect content, or enforce budgets until you enable those
features.

This page assumes only that you have called a model API from code. If you want to see TokenTriage
running first, the [Quickstart](/docs/get-started/quickstart) uses built-in mock providers and
requires no provider key, configuration file, or TokenTriage account.

## Where it sits in your stack

A **gateway** is a shared stop between an application and another service. Model requests go to
the gateway first; the gateway passes them to the provider. This lets you add accounting in one
place instead of rebuilding it in every application.

<GatewayPositionDiagram />

Read the diagram from left to right. With direct calls, each application needs its own
instrumentation if you want to connect provider spending to individual requests. With
TokenTriage in the middle, the same traffic passes through one process that records each request
as it happens.

Your provider accounts, credentials, and bills remain yours. TokenTriage is not a reseller or a
hosted model service. The ledger write happens outside the request path, so your application does
not wait for the accounting step. The gateway itself is still on the request path, so treat its
availability as you would any other proxy. The [deployment guide](/docs/operate/deployment) covers
the available topologies.

## What happens by default

A fresh installation has a narrow job: forward the request, return the response, and record its
token use and cost.

- **Your app changes one setting.** Point its model API client's base URL at TokenTriage instead
  of the provider.
- **Your provider relationship stays the same.** You use your own account and credentials, and
  continue paying the provider directly.
- **Your client receives the provider's response body.** TokenTriage adds an
  `x-tt-request-id` response header that links the request to its ledger record.
- **TokenTriage writes one local record.** The default file is
  `~/.tokentriage/decisions.jsonl`; each row carries the model, token counts, pricing status, and
  calculated cost when a price is available.

TokenTriage makes a few small changes to requests and headers so it can associate responses with
ledger records and collect token counts from streaming calls. The [HTTP headers
reference](/docs/reference/headers) documents the wire-level behavior. You do not need those
details to start using the ledger.

In `ledger` mode, an internal processing error falls back to a direct provider pass-through and is
written to the logs. That protects a provider response from an accounting failure. TokenTriage is
still a network hop: if the process or its network path is unavailable, requests cannot pass
through it.

## What the ledger lets you answer

One request record gives you a model, token counts, a pricing result, and a request ID. Add labels,
called **dimensions**, for a feature, tenant, or agent, and the same records can answer questions
the provider invoice cannot:

- Which feature or tenant is responsible for the most spending?
- Did retries or unusually large prompts cause a cost spike?
- How does spending change across models or over time?
- Which group of requests is worth evaluating on a cheaper model?

Those optional dimensions do not change the provider request or response. They give you labels to
group by on the command line, dashboard, and exports.

Routing, response caching, governance, guardrails, and evaluation are separate features. Each is
off when you start and has its own configuration. [Features at a glance](/docs/introduction/features)
shows what each feature does and where to read more.

## Is TokenTriage a fit?

TokenTriage is a good fit when:

- several features, tenants, or agents share a provider account and you need to attribute their
  costs;
- you want evidence before moving some requests to a cheaper model; or
- you prefer to run the gateway yourself instead of sending prompts through another hosted
  service.

It is probably not the right fit when:

- you want a hosted URL and no infrastructure of your own;
- you need a provider or endpoint that TokenTriage does not currently cover — check
  [Providers and dialects](/docs/build/providers) before adopting it; or
- you need software with years of production history. TokenTriage is beta.

## What you need to run it

For your own traffic, you need:

- your existing provider account and credentials;
- an application or model client whose base URL you can configure; and
- somewhere to run the TokenTriage binary.

The default ledger setup needs no TokenTriage account, database server, or language runtime. You
keep paying your providers as you do now. If you build from source instead of using a release
binary, you also need Go.

This page stops at the basic mental model. The pages below cover the runnable first experience,
the wider gateway vocabulary, optional capabilities, and product trade-offs.

<NextSteps items={[
  {
    title: "Quickstart",
    href: "/docs/get-started/quickstart",
    description: "See the core workflow locally with mock traffic, without choosing a provider or spending money."
  },
  {
    title: "LLM gateways, proxies, and routers",
    href: "/docs/introduction/llm-gateways-explained",
    description: "Build the wider mental model for where gateways fit and how they differ from routers and observability tools."
  },
  {
    title: "Features at a glance",
    href: "/docs/introduction/features",
    description: "See which capabilities are available, which are on by default, and where each one is documented."
  },
  {
    title: "How TokenTriage compares",
    href: "/docs/introduction/how-tokentriage-compares",
    description: "Compare the self-hosted approach with hosted gateways and broader provider catalogs."
  }
]} />