# Sizing and capacity
Size the volume from a measured growth rate, size the rest from what the chart suggests, and know which of those two is a measurement.
Source: https://tokentriage.com/docs/operate/capacity
---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
Capacity planning for a gateway usually means guessing at CPU. Here the dominant resource is
storage, because TokenTriage's job is to keep a durable record of every request, and that record
is the only thing on the machine that grows without you doing anything.
This page separates what has been measured from what has not. The storage arithmetic is
measured and you can reproduce it. The compute figures are not, and rather than manufacture
them, this page says so and points at the chart's starting point.
## The one number to derive yourself
Ledger size is the product of three things you control: how many requests you take, how large a
row is for your traffic, and how long you keep the history.
```text
active file ≈ rotate_mb (bounded, by construction)
history ≈ requests × row_bytes ÷ 53 (compressed companions)
volume ≈ rotate_mb + history + headroom
```
The `53` is a compression ratio measured on real rows, and the section after next explains where
it comes from and when to distrust it.
Measure `row_bytes` on your own traffic rather than adopting a figure. Twenty single-message
Anthropic completions through a local mock gave this:
```console
$ wc -c -l data/decisions.jsonl
20 34620 data/decisions.jsonl
```
**1,731 bytes per row.** Rows grow with the number of messages, tool definitions and business
dimensions you attach, and grow again in `shadow` mode, which records the would-route decision
alongside the actual one. A row costed against a model with cache tiers carries a few more
fields. None of that changes the shape of the arithmetic, only the constant.
At 1.7 KB a row, one million requests is about **1.6 GiB** of raw ledger and roughly **31 MiB**
once rotated and compressed. The chart's default `10Gi` volume holds an unrotated corpus of
about six million rows, and vastly more once history compresses.
## Worked: a hundred thousand requests a day
Take a deployment serving 100,000 requests a day with rows of the measured size, keeping ninety
days of history.
| Quantity | Value | Where it comes from |
|---|---|---|
| rows per day | 100,000 | your traffic |
| raw bytes per day | ~165 MiB | 100,000 × 1,731 |
| compressed per day | ~3.2 MiB | ÷ 53 |
| ninety days of history | ~280 MiB | × 90 |
| active file | ≤ 256 MiB | `rotate_mb` default |
| volume | **1 GiB is comfortable** | sum, doubled for headroom |
The striking part is that the *bounded* active file dominates a ninety-day history. That is the
right instinct to carry away: rotation, not retention, is what keeps a ledger volume small, and
history is nearly free.
Set the policy that produces the second row of that table:
```yaml
ledger:
decision_log:
rotate_mb: 256 # default; bounds the active file
retention:
max_age: 2160h # ninety days
max_total_mb: 512 # a ceiling regardless of age
```
Both retention limits default to `0`, which means keep forever, and the sweeper only runs when
the dashboard is enabled. [The decision ledger on
disk](/docs/operate/ledger-lifecycle) covers both traps.
## Where the compression ratio comes from
Driving 2,500 requests through an instance with `rotate_mb: 1` produced four companions:
```console
$ ls -la data/
-rw-r--r-- 1 19668 decisions-20260810T125338-1.jsonl.gz
-rw-r--r-- 1 19588 decisions-20260810T125338-2.jsonl.gz
-rw-r--r-- 1 19635 decisions-20260810T125338-3.jsonl.gz
-rw-r--r-- 1 19647 decisions-20260810T125338.jsonl.gz
-rw-r--r-- 1 159192 decisions.jsonl
```
Each holds a full mebibyte of rows in about 19.6 KB. Divide 1,048,576 by 19,647 and the ratio is **53.4**.
Distrust that ratio when your traffic is less uniform than the run behind it. Those rows shared
one model, one prompt shape and one price, which is close to the best case for gzip on JSON. A
mixed workload across many models with varied dimensions compresses less. The number is worth
using as a planning estimate and worth re-measuring from your own companions once you have a
week of them:
```console
$ ls -la /data/.tokentriage/*.jsonl.gz
```
## What saturates first
Three things can become the constraint, and they fail in different ways.
**Storage** is the one that ends the deployment. A full volume stops the writer, and the writer
drops rather than blocking, so traffic keeps flowing while the record of it stops. That failure
is silent on every surface except a counter. Alert on free bytes, because nothing ships that
does.
**The writer queue** is the early warning for storage. It holds 8,192 rows by default and drops
past that, counting each one. A moving `tokentriage_declog_dropped_total` on an instance whose
disk is not yet full means the volume is slow rather than small.
**CPU** is what the horizontal autoscaler watches, and the reason to add replicas rather than
grow one. Adding replicas is not free: it requires the shared-state tier so governance stays
coordinated, and each replica keeps its own ledger on its own volume, so your storage arithmetic
multiplies by N. [Autoscaling](/docs/operate/autoscaling) covers the gate and the transient.
## What is not measured
Three figures an operator reasonably wants are not published here, and inventing them would be
worse than the gap.
**Steady-state memory.** No figure is published. The chart's suggested starting point —
`requests: {cpu: 250m, memory: 256Mi}`, `limits: {cpu: "1", memory: 512Mi}` — is guidance for a
single-node install rather than a measurement of one, and `resources` is empty by default so the chart
imposes nothing you did not choose. Set requests from what your own deployment uses under load,
and treat the suggestion as a place to start.
**CPU per request.** Not published as a per-request figure. What has been measured, on bare
metal under a documented harness, is the gateway's *added latency*: the overhead TokenTriage
contributes on top of the provider call. That is a different quantity from CPU cost and it lives
on the [benchmarks](/docs/reference/benchmarks) page with its conditions.
**File-descriptor behaviour.** Not measured. The proxy holds connections to upstreams plus the
ledger and index handles; no published figure bounds that against concurrency. Size your `ulimit`
generously and watch it rather than trusting an estimate from us.
Do not derive a capacity plan from the benchmark numbers. Those runs measured added latency and
throughput under a specific harness on specific hardware; they were not a sizing exercise, and
the phrasing rules that govern them exist because the two get confused. Size storage from the
arithmetic above and compute from your own observation.
The deployed artifact is one static binary in a `FROM scratch` image, with no base layer, no shell and
no package manager. The layers, from a local build of the compose image:
| Layer | Size |
|---|---|
| the binary | 38.6 MB |
| vendored license texts | 1.85 MB |
| CA certificate bundle | 197 kB |
| baked config, user database, data dir | ~37 kB |
That binary is from a plain `go build` with debug information present; the release pipeline
strips it, so a published image is smaller. Both figures are far below the point where image
size drives a mirroring decision. [Install air-gapped](/docs/operate/air-gap) covers what
crosses a gap in practice.