Tokens, prices, and where the money goes

A provider invoice gives you one total for the month. In that month, your application may have made a thousand calls or a million, across several models and features. The invoice does not connect that total to the work your application performed.

It cannot tell you which agent spent the money, which feature was expensive, or whether a retry loop caused a spike. A monthly total does not explain where the money went.

To answer those questions, you need to price each request and attach the labels you care about. This page explains the four ideas behind that:

• Providers turn text into tokens before a model processes it.
• Input, output, and cached tokens can have different rates.
• A request’s cost is each token count multiplied by its rate, then added together.
• A trustworthy cost records which price list was used and reports an unknown price as unknown, not zero.

You don’t need to have installed anything to read it. If you already price requests by hand and want the field-level breakdown, skip ahead to the ledger field reference.

What a token is, and why output often costs more

For text generation, providers usually quote prices per token, so it helps to understand what the unit represents.

A token is a small piece of text as the model sees it: a short word, part of a longer word, or a piece of punctuation. A tokenizer converts your prompt into those pieces before the model processes it. Providers use the resulting counts as one of the main inputs to request cost.

A typical text-generation call has two sides, and they carry different rates.

• Input tokens are what you send: system prompt, conversation history, retrieved documents, tool definitions, the user’s message.
• Output tokens are what the model generates in reply.

Output often costs more than input. For openai/gpt-5 in the price table TokenTriage ships, input is $1.25 per million tokens and output is $10.00 per million: eight times the rate.

Providers quote those rates per million tokens, written $/MTok, and quote them separately for input, cached input, and output. To price one request, multiply each count by its rate and divide by one million.

Two practical consequences follow:

• A model that generates more output tokens costs more at the same output rate.
• In a chat application, each turn often includes previous messages as input. The input can grow even when the newest user message is short.

The other token classes you will meet

Provider usage reports may contain more than input and output. Three other token classes appear often enough to recognize:

Cache read. A provider may reuse an already-processed prompt prefix and charge a lower rate for it. For openai/gpt-5, the shipped table prices cached input at $0.125/MTok, compared with $1.25/MTok for fresh input.

Cache write. Some providers charge to create a cached prompt prefix. Later reads must save enough to repay that write before the entry expires.

Reasoning tokens. Some models use additional tokens before producing the visible answer. Those tokens are usually not returned to your application. Depending on the model and provider, they may appear separately in usage reports and be billed.

Whether a cache read happens at all is the provider’s rule, not yours. OpenAI, for example, caches a prompt prefix only once it reaches 1,024 tokens (prompt caching). Minimums, entry lifetimes, and write surcharges differ from provider to provider, so check the docs for the one you’re calling rather than carrying one vendor’s numbers across.

TokenTriage prices each class against its own rate rather than folding everything into one token count. It also records the classes separately on the request’s ledger row. The ledger field reference lists the exact field names and how each one is populated.

Prompt caching and response caching are different

A provider prompt-cache hit is a discount on a call that still happens. The provider processes the request but charges less for a prefix it has seen before.

A response-cache hit avoids the provider call. Something in front of the provider already has the answer and returns it directly. TokenTriage can do this with its own response cache, which is off until you enable it.

Both can save money, but they measure different events. TokenTriage keeps them separate so the same savings are not counted twice. Two caches and how their savings stay apart explains the accounting and break-even rules.

One request, priced by hand

You can verify a per-request price yourself with a calculator.

TokenTriage writes one row per proxied request to ~/.tokentriage/decisions.jsonl, and top rolls those rows up. Consider one gpt-5 call with a 1,280-token prompt. The provider reports 1,024 of those tokens as a cache read and 256 as fresh input. The model then generates 40 output tokens.

On the TokenTriage row, the fresh and cached input are separate counts:

fresh input      256 tokens
cached input   1,024 tokens
output             40 tokens

The full top table also includes request counts, cache share, and latency percentiles. Those fields are useful operationally, but they are not needed for this calculation.

Using the provider’s per-million rates directly, the arithmetic is:

  256 fresh-input tokens  x   $1.25 / 1,000,000  =  $0.000320
   40 output tokens       x  $10.00 / 1,000,000  =  $0.000400
1,024 cached-input tokens x  $0.125 / 1,000,000  =  $0.000128
                                                   ─────────
                                                   $0.000848

Given those counts and rates, the result is exact rather than heuristic. Redo the calculation and you should get the same answer. If you do not, there is a specific discrepancy to investigate.

Why a price needs a date

Providers change prices as models are repriced, tiers are renamed, and discounts are added. The same request can cost a different amount after a rate changes.

Every cost TokenTriage writes therefore carries the price list it was computed against in a pricing_snapshot field. This build uses litellm@2026-07-14#179affb. If two reports differ, the snapshot helps you determine whether the traffic changed or the price list did. Where every number comes from explains the full provenance model.

Unknown cost is not zero

Sometimes TokenTriage cannot find a per-token rate. The model may be missing from the price table, or the provider may bill that kind of request per image, second, or character instead of per token.

In those cases, $0.00 would be misleading. It means “this request cost nothing,” while the actual answer is “the cost is unknown.” Treating an unknown price as zero makes totals look lower than they may really be.

TokenTriage marks the row unpriced and prints no dollar figure: - in CLI tables and null over the API. Totals exclude those rows and report how many were excluded. A model you explicitly declare free receives a real $0 with a provenance stamp.

The rules for this, including the typed reasons a lookup can miss, are in provenance and unpriced rows.

What a checkable dollar buys you

Once every request carries a price you can verify and the labels you attach, you can answer questions the provider invoice cannot:

• Which agent, feature, or customer generated the cost?
• Did retries cause a spike after a release?
• Which models account for most of the total?
• Could a cheaper model meet the same quality target?

top --by model is the smallest version of that. The same rollup works by agent and by the other labels you send on the request, so you can cost a multi-step agent one step at a time instead of as one total. Cost analytics covers the reporting surfaces, and attribute cost per agent step is a worked example that takes an agent run apart.