# Providers that take a job and answer later
Why a submitted job lands on the ledger unpriced, what arrives later to carry its dollar, and how an abandoned job is recorded.
Source: https://tokentriage.com/docs/build/providers-async-jobs
---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
import Screenshot from '../../../components/Screenshot.astro';
import AsyncSettlementDiagram from '../../../components/AsyncSettlementDiagram.astro';
import JobLifecycleDiagram from '../../../components/JobLifecycleDiagram.astro';
You asked an image model for a picture. It answered `202 Accepted` and a job id instead of an
image, and the cost line for that request reads `UNPRICED`. Nothing has gone wrong. The provider
has not finished the work, so nobody knows yet what the work cost.
A dollar does arrive. It arrives on a second line, minutes to tens of minutes later, and it
replaces the zero rather than being added to it. Everything in between is written down: the job's
identity, every poll attempt, and which of five endings it reached.
`UNPRICED` is how a row records that no dollar figure was resolved, together with a typed reason
saying why. The other reasons are covered in
[which pricing key your model lands under](/docs/build/providers-pricing-keys); the two settlement
reasons are this page's subject. If you already know how the mechanism works and what you want is
the month-end arithmetic, go to [reading a settled cost](/docs/build/providers-settlement) instead.
## A job is two rows, not one
A synchronous request produces one ledger row, and that row carries its cost. An async request
produces two, and only the second one carries money. Read the first row as a complete record and
you will conclude either that the call was free or that pricing is broken.
The submit itself is handled like any other request. It is relayed byte for byte, admitted through
the same scope, rate and concurrency limits everything else goes through, and written to the ledger
the moment it finishes. What it cannot carry is a cost: the provider answered with an
acknowledgement rather than a result.
So the row is terminal and unpriced, with the typed reason `settlement_pending`. The `job_id` on it
is the provider's own handle; TokenTriage mints none, which is why you can take that id straight
off the row and poll the provider yourself.
Here is the shape as a timeline, with both ledger rows on it:
Read it left to right, along one request's life. The left half is the submit instant, where line 1
lands carrying a zero it does not claim to believe. The right half is the settlement instant,
minutes later, where line 2 is appended and joined to line 1 by `req_id`. The rule along the bottom
is the one to carry away: a row's cost is its settlement's when one exists, and its own when none
does.
This is the cost cell of a submit row, lifted out of one:
```json
{"usd":0,"pricing_key":"","pricing_snapshot":"litellm@2026-07-14#179affb","estimated":true,"would_usd":null,"unpriced":true,"unpriced_reason":"settlement_pending"}
```
`unpriced` is the field that decides how every reader treats the row. The empty `pricing_key` is
the interesting part: a pending job has no winning key yet, and inventing one would attach
provenance to a dollar that does not exist.
Later, when the poller sees a terminal status, a second line is appended and joined to the first by
`req_id`:
```json
{"v":1,"kind":"settlement","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","ts":"2026-08-25T03:36:56.604Z","provider":"openai","endpoint":"chat","job_id":"job_abc123","status":"succeeded","attempts":3,"usage":{"input_tokens":1000,"output_tokens":500,"cache_read_tokens":0,"cache_write_tokens":0,"reasoning_tokens":0,"total_tokens":1500,"source":"provider"},"cost":{"usd":0.00045,"pricing_key":"openai/gpt-4o-mini","pricing_snapshot":"litellm@2026-07-14#179affb","estimated":false,"would_usd":null}}
```
That figure checks by hand: 1,000 input tokens at $0.00000015 plus 500 output tokens at $0.0000006
is $0.00045. `usage.source` is `absent` on the submit row and `provider` on the settlement, so
neither end guessed a token count. The ledger is append-only, so the submit row is not corrected in
place; the settlement is a second line beside it, and the dashboard's record drawer shows both.
Both blocks came out of one run of the product's async end-to-end test
(`go test ./test/e2e/ -run '^TestJobSubmitRowPendingThenSettles$' -v -count=1`) at commit `3693f7bd`,
against a mock shaped like an async provider rather than a live one, with the
`litellm@2026-07-14#179affb` price table. The `req_id`, the timestamps and the 31 milliseconds
between them belong to that machine; the field names, the two `unpriced` values and the shared
`req_id` are what you will see. Against a live provider the gap is minutes to hours.
```console
$ go test ./test/e2e/ -run '^TestJobSubmitRowPendingThenSettles$' -v -count=1
=== RUN TestJobSubmitRowPendingThenSettles
jobs_settlement_test.go:454: ★ ADJACENT LEDGER LINES (D:\tt-w01-neutral\tmp\TestJobSubmitRowPendingThenSettles777884053\001\decisions.jsonl):
jobs_settlement_test.go:455: [pending] {"v":1,"ts":"2026-08-25T03:36:56.573Z","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","trace":{"trace_id":"","step_id":"","parent_id":"","agent":"","step_class":"user_turn","step_class_source":"derived","w3c":"","source":"none"},"http":{"path":"/v1/async/generate","dialect":"openai","stream":false,"status":202,"endpoint":"chat","provider":"openai"},"request":{"model":"gpt-4o-mini","prompt_sha256":"","prompt_bytes":0,"messages":0,"system_bytes":0,"has_tools":false,"tool_count":0,"tool_result_count":0,"max_tokens":0},"signals":{},"decision":{"mode":"ledger","tier":"","upstream":"up","model_rewrite":null,"rule":"","candidates":null,"shadow_tier":null,"override":false,"fail_safe":false,"affinity_pin":false,"cooldown_skips":[]},"usage":{"input_tokens":0,"output_tokens":0,"cache_read_tokens":0,"cache_write_tokens":0,"reasoning_tokens":0,"total_tokens":0,"source":"absent"},"cost":{"usd":0,"pricing_key":"","pricing_snapshot":"litellm@2026-07-14#179affb","estimated":true,"would_usd":null,"unpriced":true,"unpriced_reason":"settlement_pending"},"latency":{"ttft_ms":0,"total_ms":0,"overhead_us":0},"outcome":{"finish_reason":"","error":"","client_disconnect":false,"signals":{"exhaustion":false,"tool_failure":false,"tool_loop":false,"user_disengagement":false,"user_satisfaction":false},"oracle":null},"mcp":[]}
jobs_settlement_test.go:456: [settlement] {"v":1,"kind":"settlement","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","ts":"2026-08-25T03:36:56.604Z","provider":"openai","endpoint":"chat","job_id":"job_abc123","status":"succeeded","attempts":3,"usage":{"input_tokens":1000,"output_tokens":500,"cache_read_tokens":0,"cache_write_tokens":0,"reasoning_tokens":0,"total_tokens":1500,"source":"provider"},"cost":{"usd":0.00045,"pricing_key":"openai/gpt-4o-mini","pricing_snapshot":"litellm@2026-07-14#179affb","estimated":false,"would_usd":null}}
jobs_settlement_test.go:457: ★ JOB JOURNAL (D:\tt-w01-neutral\tmp\TestJobSubmitRowPendingThenSettles777884053\001\jobs\jobs.jsonl):
{"v":1,"kind":"job","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","upstream":"up","provider":"openai","endpoint":"chat","model":"gpt-4o-mini","job_id":"job_abc123","poll_path":"/v1/async/generate/job_abc123","submitted":"2026-08-25T03:36:56.573Z"}
{"v":1,"kind":"settled","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","ts":"2026-08-25T03:36:56.604Z","status":"succeeded","attempts":3}
jobs_settlement_test.go:458: ★ POLLS SERVED BY THE MOCK: 3 (2 processing + 1 terminal); poll Authorization was present and is in NEITHER file
--- PASS: TestJobSubmitRowPendingThenSettles (0.05s)
PASS
ok github.com/tokentriage/platform/test/e2e 0.105s
```
The temp directory is the test's, under whatever `TMP` the run had; this capture used a scratch one
so no personal path is baked in. The two filenames, `decisions.jsonl` and `jobs/jobs.jsonl`, are
yours as well as ours. The last line is a credential check: the poll carried an `Authorization`
header, and it appears in neither file.
## What a pending row does to your totals
A pending row is a row with no money on it, and there are two dishonest ways to handle that. Drop
it, and the bill is smaller than what you were charged with nothing saying so. Sum its zero, and
you have made a confident claim about spend nobody has measured. Every cost surface here does the
third thing: it excludes the row from the money and prints the count it excluded.
```console
$ tokentriage top --by model --log D:\tt-w01-neutral\work\t3-seeded-decisions.jsonl
MODEL REQS INPUT OUTPUT CACHE-R CACHE-W REASON CACHE-SAV COST P50 P90 P99
gpt-4o-mini 5 4600 1400 0 0 0 0.0% $0.001530 $0.000360 $0.000450 $0.000450
TOTAL 5 4600 1400 0 0 0 0.0% $0.001530
COST excludes 1 unpriced row of 5 (no price resolved; the $0 is a missing price, not $0 of spend)
pending settlement: 1 row (an async job the provider has not costed yet; a settlement row will resolve it — the dollars are not missing, they are not knowable yet)
```
That ledger holds five requests: three synchronous chat calls, one job that settled, and one job
still in flight. The `--log` path and the ids inside the file belong to the capture; every number in
the table follows from those six lines and the vendored price table. The two lines under the table
are conditional, so a ledger with no unpriced rows and no async jobs renders without them.
Three things follow, and they are the ones people get wrong:
- **$0.001530 is three synchronous rows at $0.00036 plus the settled job's $0.00045.** The settled
job contributes its settlement's dollar, and the submit row it resolves carries `usd: 0`, so
adding the two together moves no money. That is what makes this one hard to catch: the fold
replaces the row's standing rather than its figure, and the double-count lands in the population
instead. Count a submit row and its settlement as two requests and this ledger reports six where
it has five.
- **`pending settlement: 1 row` is a subset of the unpriced count, not a sibling.** This ledger has
one unpriced row and it is that same row. Summing the two disclosure lines double-counts it.
- **`REQS` is 5 while `COST` covers 4.** The table does not quietly shrink its denominator to the
rows it could price, and a group with requests but no priced row among them renders `-` rather
than a confident zero.
The dashboard applies the same rule to the same ledger, and shows it rather than stating it:
That capture carries the exclusion half of this page and not the settlement half: no dashboard
surface renders a settlement-specific figure yet, so `pending settlement` appears in the CLI table
above and nowhere on the screen. The ledger underneath is synthetic and every row carries
`demo: true`.
The two exclusions also mean opposite things. *No price resolved* is a catalog gap somebody has to
fix. *Awaiting settlement* is a queue finishing the figure by itself. Reconciling a month against
those rules is a procedure rather than a mental model, and it lives on
[reading a settled cost](/docs/build/providers-settlement).
## Which providers answer later
Seven provider descriptors submit work and poll for it. What they have in common is the shape
above; what differs is whether the finished job produces a dollar at all.
| Descriptor | Surfaces | What its catalog rows settle to |
|---|---|---|
| `replicate` | chat | none of its 40 rows produce a dollar; the wire reports elapsed seconds and every row is priced per token |
| `stability` | images, image edits | all 23 rows price |
| `fal_ai` | images | 9 of 14 price; 5 settle `unit_missing` |
| `black_forest_labs` | images, image edits | all 8 price, at one image per request |
| `runwayml` | images, speech | 1 of 3 prices; 2 settle `unit_missing` |
| `assemblyai` | transcription, files | the legacy tier names price; a present-day request settles `model_unknown` |
| `soniox` | transcription, files | neither of its 2 rows prices; the catalog carries no rate for them |
Four of those seven carry an end-to-end test that drives a request through the descriptor against
a mock shaped like that provider and asserts the ledger row's own provider and endpoint stamps:
`replicate`, `stability`, `assemblyai` and `soniox`. The other three (`fal_ai`,
`black_forest_labs` and `runwayml`) declare their endpoints with no such test, and the generated
matrix prints those cells as `declared (no e2e gate)`, calling them "not a support claim" rather
than hiding them. The tests that do exist run against mocks, not the providers' live endpoints.
Binding a provider means its traffic is routed, governed, journaled, polled and settled. It does
not mean every model produces a dollar, and Replicate is the clearest case: 40 catalog rows, zero
priced dollars, because the unit its price needs is absent from the wire. Coverage and priced
dollars are different claims, and anyone quoting the 40 in a spend or savings sentence is quoting
the wrong number.
Those seven used to be a class of their own in the published coverage partition, and
[the class that emptied](/docs/build/providers-coverage#the-class-that-emptied) explains why it is
still listed with a count of zero.
OpenAI's Batch API is the eighth async surface, and the odd one out. It is bound to `/v1/batches`,
the path OpenAI serves, and `batch` is not among the endpoint kinds the OpenAI descriptor declares.
Its settlement sums the output file's per-line usage, where a malformed line is excluded and
counted rather than guessed at from its neighbours. One caveat travels with every priced batch row:
settlement prices at the synchronous rate, and OpenAI publishes a batch tier at half that, so the
figure overstates by a factor of two. The row carries that disclosure instead of a guessed rate.
## What the engine does between the two rows
The gap between the two rows is where an async gateway usually loses things. A restart, a config
reload, or a provider that stops answering are all ordinary events, and each of them is a chance
for a job to vanish along with the money it stands for.
So the job plane keeps its own append-only journal beside the ledger, one JSON object per line, in
two kinds: a `job` line at submit and a `settled` line at the terminal state.
```json
{"v":1,"kind":"job","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","upstream":"up","provider":"openai","endpoint":"chat","model":"gpt-4o-mini","job_id":"job_abc123","poll_path":"/v1/async/generate/job_abc123","submitted":"2026-08-25T03:36:56.573Z"}
{"v":1,"kind":"settled","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","ts":"2026-08-25T03:36:56.604Z","status":"succeeded","attempts":3}
```
Neither line carries a credential, and neither does the record behind them. A job is made of
identifiers and coordinates, with no field for a secret, so keeping keys out of the journal is not
something you configure.
The journal is replayed at boot, and torn, unknown or duplicate lines are skipped and counted
rather than aborting the replay. Jobs are carried across a config reload by identity, so editing
your config does not rebuild the queue underneath it, and a job whose upstream you deleted keeps
polling until it hits the attempt cap. One poller drives all of them: each provider supplies its
own submit step and its own poll step, and the schedule, the concurrency limit, the attempt cap and
the writing of the settlement row are shared.
Every journaled job reaches exactly one ending, and the set of endings is closed:
Four of the five come from the provider: `succeeded`, `failed`, `canceled`, `expired`. The fifth,
`abandoned`, is written by the engine when a job exhausts its attempt cap, and no driver can
declare it. A settlement whose status is not `succeeded` prices nothing and says so, with the typed
reason `settlement_terminal`.
Abandonment is the one worth dwelling on, because in most systems it is an absence: a job stops
being polled and nothing is written anywhere. Here it is a record. The engine appends a terminal
settlement row carrying the attempt count and the detail `poll attempt cap (60) exhausted`, plus
the last error where there was one, redacted. An unrecognised state from a driver is not coerced
into one of the five either. It is counted, disclosed, and polling continues within the cap, and if
the job does end up abandoned, the abandonment quotes the value it could not read.
The plane is off unless you turn it on, and off means inert: no poller goroutine, no `jobs` key on
the status endpoint, and a byte-identical ledger.
```yaml
jobs:
enabled: true
```
Everything else has a default worth knowing before you change one.
| Key | Default | What it decides |
|---|---|---|
| `jobs.state_dir` | the ledger directory's `jobs` folder | where the journal is written |
| `jobs.poll.max_attempts` | `60` | attempts before a job is abandoned |
| `jobs.poll.max_concurrent` | `4` | how many polls run at once |
| `jobs.poll.backoff.initial` | `2s` | when the first poll happens |
| `jobs.poll.backoff.max` | `60s` | the ceiling the interval doubles towards |
| `jobs.retention.settled` | `720h`, and an explicit `0s` keeps them forever | how long settled entries stay in the journal |
The default journal path is two `jobs` segments deep, which surprises people looking for it: a
state directory named `jobs` beside the ledger, and `jobs/jobs.jsonl` inside that. The `jobs plane
armed` line at boot prints the full path, which is the quickest way to confirm it.
## When the settlement arrives
The schedule is fixed and knowable, which means the worst case is a number rather than a feeling.
The first poll happens two seconds after the submit, with no jitter on it. Each interval after that
doubles until it reaches the 60-second ceiling, and each one is multiplied by a jitter factor that
is fixed for the life of the job and derived from its request id, landing somewhere in
`[0.8, 1.2]`. Sixty attempts at those intervals sum to `2s + f × 3360s`.
| Jitter factor | A job that reaches no terminal state is abandoned at |
|---|---|
| `0.8`, the floor | about 45 minutes |
| `1.0` | about 56 minutes |
| `1.2`, the ceiling | about 67 minutes |
**Plan for roughly 56 minutes, and treat 45 to 67 as the band.** All three are lower bounds: the
clock for the next interval starts when a poll finishes, so each attempt's own round trip, up to 30
seconds of it, is added on top. `jobs.poll.max_attempts` and `jobs.poll.backoff.max` are the two
knobs that move the number.
That horizon is also the bound on budget enforcement, which is worth planning for rather than
discovering. Admission gates an async submit like any other request: scope, rate and concurrency
limits all apply the moment the job is created. A spend gate is different. It sees the job's cost
only when the settlement row reaches the ledger, minutes to tens of minutes later.
No reservation against a predicted job cost is attempted at any point. A budget that turned a
request away against an estimate of what an image generation might cost would be enforcing a figure
nobody measured, so the exposure between submit and settlement is left open and disclosed rather
than papered over with a guess. The disclosure is a field: every budget status carries
`pending_settlement_rows` beside `unpriced_rows`, and a rising pending count beside a flat burn is
the picture of a budget that has not yet been told about spend that has already happened. Alert on
that field, not on burn alone.
One trap in the other direction: shortening `max_attempts` does not make the money arrive sooner.
It converts a job that would have settled into one that is abandoned, which is a written outcome
rather than a silent forgetting, but also a job whose cost you will not learn.
## Three things this does not do
### There is no synchronous facade
A wrapper that would have accepted an OpenAI-format `/v1/images/generations`, polled the provider
internally, and answered with the finished image was planned, sequenced last, and dropped. What it
would have sat on shipped in full: the journal, the poller, the settlement row, the seven
providers. So your client still speaks the provider's own async contract. Submit, take the
provider's own job id, poll the provider's own status path through the gateway. We do not hide the
wait for you.
`jobs.facade.enabled: true` passes validation and mounts nothing. `tokentriage validate` prints
`OK` and exits `0`, a run mounts no route, and no log line mentions the field. The config schema
was frozen ahead of the decision to drop it, so the key is accepted and range-checked and then read
by nobody. Do not read that `OK` as a feature: own the poll loop in your own code, and read the
settlement row for the money.
### There is no `tokentriage jobs` command
The job plane is observable in three places and no others. The `jobs` block on `GET /tt/api/status`
appears when the plane is armed and is omitted when it is not. A `jobs plane armed` line at boot
carries the journal path and how many pending jobs this process adopted from the last one. A `jobs
settlement summary` line at shutdown carries what was carried forward, attempted, settled and
abandoned. Pending jobs at shutdown are not a failure; they stay in the journal and are re-polled
at the next boot.
### The capabilities endpoint does not mention the poller
`GET /tt/api/v1/capabilities` carries no jobs, async, settlement or poller field at all. Its one
egress-related field, `judge_external_dependency`, is computed from the evals evaluators and the
shadow-duplication block and nothing else. A deployment with the job plane armed and neither of
those configured reports `judge_external_dependency: false` while the poller is dialling a provider
out of band.
Read `jobs.enabled` in the config, and read the `jobs` block on `GET /tt/api/status`, which is
present when the plane is armed. On an air-gapped deployment the control is to leave `jobs.enabled`
off, at which point nothing polls, nothing dials out, and not one ledger row changes. This gap is
filed against the product.