Providers that take a job and answer later

You asked an image model for a picture. It answered 202 Accepted and a job id instead of an image, and the cost line for that request reads UNPRICED. Nothing has gone wrong. The provider has not finished the work, so nobody knows yet what the work cost.

A dollar does arrive. It arrives on a second line, minutes to tens of minutes later, and it replaces the zero rather than being added to it. Everything in between is written down: the job’s identity, every poll attempt, and which of five endings it reached.

UNPRICED is how a row records that no dollar figure was resolved, together with a typed reason saying why. The other reasons are covered in which pricing key your model lands under; the two settlement reasons are this page’s subject. If you already know how the mechanism works and what you want is the month-end arithmetic, go to reading a settled cost instead.

A job is two rows, not one

A synchronous request produces one ledger row, and that row carries its cost. An async request produces two, and only the second one carries money. Read the first row as a complete record and you will conclude either that the call was free or that pricing is broken.

The submit itself is handled like any other request. It is relayed byte for byte, admitted through the same scope, rate and concurrency limits everything else goes through, and written to the ledger the moment it finishes. What it cannot carry is a cost: the provider answered with an acknowledgement rather than a result. So the row is terminal and unpriced, with the typed reason settlement_pending. The job_id on it is the provider’s own handle; TokenTriage mints none, which is why you can take that id straight off the row and poll the provider yourself.

Here is the shape as a timeline, with both ledger rows on it:

Read it left to right, along one request’s life. The left half is the submit instant, where line 1 lands carrying a zero it does not claim to believe. The right half is the settlement instant, minutes later, where line 2 is appended and joined to line 1 by req_id. The rule along the bottom is the one to carry away: a row’s cost is its settlement’s when one exists, and its own when none does.

This is the cost cell of a submit row, lifted out of one:

{"usd":0,"pricing_key":"","pricing_snapshot":"litellm@2026-07-14#179affb","estimated":true,"would_usd":null,"unpriced":true,"unpriced_reason":"settlement_pending"}

unpriced is the field that decides how every reader treats the row. The empty pricing_key is the interesting part: a pending job has no winning key yet, and inventing one would attach provenance to a dollar that does not exist.

Later, when the poller sees a terminal status, a second line is appended and joined to the first by req_id:

{"v":1,"kind":"settlement","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","ts":"2026-08-25T03:36:56.604Z","provider":"openai","endpoint":"chat","job_id":"job_abc123","status":"succeeded","attempts":3,"usage":{"input_tokens":1000,"output_tokens":500,"cache_read_tokens":0,"cache_write_tokens":0,"reasoning_tokens":0,"total_tokens":1500,"source":"provider"},"cost":{"usd":0.00045,"pricing_key":"openai/gpt-4o-mini","pricing_snapshot":"litellm@2026-07-14#179affb","estimated":false,"would_usd":null}}

That figure checks by hand: 1,000 input tokens at $0.00000015 plus 500 output tokens at $0.0000006 is $0.00045. usage.source is absent on the submit row and provider on the settlement, so neither end guessed a token count. The ledger is append-only, so the submit row is not corrected in place; the settlement is a second line beside it, and the dashboard’s record drawer shows both.

Both blocks came out of one run of the product’s async end-to-end test (go test ./test/e2e/ -run '^TestJobSubmitRowPendingThenSettles$' -v -count=1) at commit 3693f7bd, against a mock shaped like an async provider rather than a live one, with the litellm@2026-07-14#179affb price table. The req_id, the timestamps and the 31 milliseconds between them belong to that machine; the field names, the two unpriced values and the shared req_id are what you will see. Against a live provider the gap is minutes to hours.

$ go test ./test/e2e/ -run '^TestJobSubmitRowPendingThenSettles$' -v -count=1
=== RUN   TestJobSubmitRowPendingThenSettles
    jobs_settlement_test.go:454: ★ ADJACENT LEDGER LINES (D:\tt-w01-neutral\tmp\TestJobSubmitRowPendingThenSettles777884053\001\decisions.jsonl):
    jobs_settlement_test.go:455:   [pending]    {"v":1,"ts":"2026-08-25T03:36:56.573Z","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","trace":{"trace_id":"","step_id":"","parent_id":"","agent":"","step_class":"user_turn","step_class_source":"derived","w3c":"","source":"none"},"http":{"path":"/v1/async/generate","dialect":"openai","stream":false,"status":202,"endpoint":"chat","provider":"openai"},"request":{"model":"gpt-4o-mini","prompt_sha256":"","prompt_bytes":0,"messages":0,"system_bytes":0,"has_tools":false,"tool_count":0,"tool_result_count":0,"max_tokens":0},"signals":{},"decision":{"mode":"ledger","tier":"","upstream":"up","model_rewrite":null,"rule":"","candidates":null,"shadow_tier":null,"override":false,"fail_safe":false,"affinity_pin":false,"cooldown_skips":[]},"usage":{"input_tokens":0,"output_tokens":0,"cache_read_tokens":0,"cache_write_tokens":0,"reasoning_tokens":0,"total_tokens":0,"source":"absent"},"cost":{"usd":0,"pricing_key":"","pricing_snapshot":"litellm@2026-07-14#179affb","estimated":true,"would_usd":null,"unpriced":true,"unpriced_reason":"settlement_pending"},"latency":{"ttft_ms":0,"total_ms":0,"overhead_us":0},"outcome":{"finish_reason":"","error":"","client_disconnect":false,"signals":{"exhaustion":false,"tool_failure":false,"tool_loop":false,"user_disengagement":false,"user_satisfaction":false},"oracle":null},"mcp":[]}
    jobs_settlement_test.go:456:   [settlement] {"v":1,"kind":"settlement","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","ts":"2026-08-25T03:36:56.604Z","provider":"openai","endpoint":"chat","job_id":"job_abc123","status":"succeeded","attempts":3,"usage":{"input_tokens":1000,"output_tokens":500,"cache_read_tokens":0,"cache_write_tokens":0,"reasoning_tokens":0,"total_tokens":1500,"source":"provider"},"cost":{"usd":0.00045,"pricing_key":"openai/gpt-4o-mini","pricing_snapshot":"litellm@2026-07-14#179affb","estimated":false,"would_usd":null}}
    jobs_settlement_test.go:457: ★ JOB JOURNAL (D:\tt-w01-neutral\tmp\TestJobSubmitRowPendingThenSettles777884053\001\jobs\jobs.jsonl):
        {"v":1,"kind":"job","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","upstream":"up","provider":"openai","endpoint":"chat","model":"gpt-4o-mini","job_id":"job_abc123","poll_path":"/v1/async/generate/job_abc123","submitted":"2026-08-25T03:36:56.573Z"}
        {"v":1,"kind":"settled","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","ts":"2026-08-25T03:36:56.604Z","status":"succeeded","attempts":3}
    jobs_settlement_test.go:458: ★ POLLS SERVED BY THE MOCK: 3 (2 processing + 1 terminal); poll Authorization was present and is in NEITHER file
--- PASS: TestJobSubmitRowPendingThenSettles (0.05s)
PASS
ok  	github.com/tokentriage/platform/test/e2e	0.105s

What a pending row does to your totals

A pending row is a row with no money on it, and there are two dishonest ways to handle that. Drop it, and the bill is smaller than what you were charged with nothing saying so. Sum its zero, and you have made a confident claim about spend nobody has measured. Every cost surface here does the third thing: it excludes the row from the money and prints the count it excluded.

$ tokentriage top --by model --log D:\tt-w01-neutral\work\t3-seeded-decisions.jsonl
MODEL        REQS  INPUT  OUTPUT  CACHE-R  CACHE-W  REASON  CACHE-SAV  COST       P50        P90        P99
gpt-4o-mini  5     4600   1400    0        0        0       0.0%       $0.001530  $0.000360  $0.000450  $0.000450
TOTAL        5     4600   1400    0        0        0       0.0%       $0.001530                        
COST excludes 1 unpriced row of 5 (no price resolved; the $0 is a missing price, not $0 of spend)
pending settlement: 1 row (an async job the provider has not costed yet; a settlement row will resolve it — the dollars are not missing, they are not knowable yet)

That ledger holds five requests: three synchronous chat calls, one job that settled, and one job still in flight. The --log path and the ids inside the file belong to the capture; every number in the table follows from those six lines and the vendored price table. The two lines under the table are conditional, so a ledger with no unpriced rows and no async jobs renders without them.

Three things follow, and they are the ones people get wrong:

• $0.001530 is three synchronous rows at $0.00036 plus the settled job’s $0.00045. The settled job contributes its settlement’s dollar, and the submit row it resolves carries usd: 0, so adding the two together moves no money. That is what makes this one hard to catch: the fold replaces the row’s standing rather than its figure, and the double-count lands in the population instead. Count a submit row and its settlement as two requests and this ledger reports six where it has five.
• pending settlement: 1 row is a subset of the unpriced count, not a sibling. This ledger has one unpriced row and it is that same row. Summing the two disclosure lines double-counts it.
• REQS is 5 while COST covers 4. The table does not quietly shrink its denominator to the rows it could price, and a group with requests but no priced row among them renders - rather than a confident zero.

The dashboard applies the same rule to the same ledger, and shows it rather than stating it:

localhost:9090/ui/costDemo data

That capture carries the exclusion half of this page and not the settlement half: no dashboard surface renders a settlement-specific figure yet, so pending settlement appears in the CLI table above and nowhere on the screen. The ledger underneath is synthetic and every row carries demo: true.

The two exclusions also mean opposite things. No price resolved is a catalog gap somebody has to fix. Awaiting settlement is a queue finishing the figure by itself. Reconciling a month against those rules is a procedure rather than a mental model, and it lives on reading a settled cost.

Which providers answer later

Seven provider descriptors submit work and poll for it. What they have in common is the shape above; what differs is whether the finished job produces a dollar at all.

Four of those seven carry an end-to-end test that drives a request through the descriptor against a mock shaped like that provider and asserts the ledger row’s own provider and endpoint stamps: replicate, stability, assemblyai and soniox. The other three (fal_ai, black_forest_labs and runwayml) declare their endpoints with no such test, and the generated matrix prints those cells as declared (no e2e gate), calling them “not a support claim” rather than hiding them. The tests that do exist run against mocks, not the providers’ live endpoints.

Binding a provider means its traffic is routed, governed, journaled, polled and settled. It does not mean every model produces a dollar, and Replicate is the clearest case: 40 catalog rows, zero priced dollars, because the unit its price needs is absent from the wire. Coverage and priced dollars are different claims, and anyone quoting the 40 in a spend or savings sentence is quoting the wrong number.

Those seven used to be a class of their own in the published coverage partition, and the class that emptied explains why it is still listed with a count of zero.

OpenAI’s Batch API is the eighth async surface, and the odd one out. It is bound to /v1/batches, the path OpenAI serves, and batch is not among the endpoint kinds the OpenAI descriptor declares. Its settlement sums the output file’s per-line usage, where a malformed line is excluded and counted rather than guessed at from its neighbours. One caveat travels with every priced batch row: settlement prices at the synchronous rate, and OpenAI publishes a batch tier at half that, so the figure overstates by a factor of two. The row carries that disclosure instead of a guessed rate.

What the engine does between the two rows

The gap between the two rows is where an async gateway usually loses things. A restart, a config reload, or a provider that stops answering are all ordinary events, and each of them is a chance for a job to vanish along with the money it stands for.

So the job plane keeps its own append-only journal beside the ledger, one JSON object per line, in two kinds: a job line at submit and a settled line at the terminal state.

{"v":1,"kind":"job","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","upstream":"up","provider":"openai","endpoint":"chat","model":"gpt-4o-mini","job_id":"job_abc123","poll_path":"/v1/async/generate/job_abc123","submitted":"2026-08-25T03:36:56.573Z"}
{"v":1,"kind":"settled","req_id":"tt_dr2r2rt44gflzf3d7ufeabi75a","ts":"2026-08-25T03:36:56.604Z","status":"succeeded","attempts":3}

Neither line carries a credential, and neither does the record behind them. A job is made of identifiers and coordinates, with no field for a secret, so keeping keys out of the journal is not something you configure.

The journal is replayed at boot, and torn, unknown or duplicate lines are skipped and counted rather than aborting the replay. Jobs are carried across a config reload by identity, so editing your config does not rebuild the queue underneath it, and a job whose upstream you deleted keeps polling until it hits the attempt cap. One poller drives all of them: each provider supplies its own submit step and its own poll step, and the schedule, the concurrency limit, the attempt cap and the writing of the settlement row are shared.

Every journaled job reaches exactly one ending, and the set of endings is closed:

Four of the five come from the provider: succeeded, failed, canceled, expired. The fifth, abandoned, is written by the engine when a job exhausts its attempt cap, and no driver can declare it. A settlement whose status is not succeeded prices nothing and says so, with the typed reason settlement_terminal.

Abandonment is the one worth dwelling on, because in most systems it is an absence: a job stops being polled and nothing is written anywhere. Here it is a record. The engine appends a terminal settlement row carrying the attempt count and the detail poll attempt cap (60) exhausted, plus the last error where there was one, redacted. An unrecognised state from a driver is not coerced into one of the five either. It is counted, disclosed, and polling continues within the cap, and if the job does end up abandoned, the abandonment quotes the value it could not read.

The plane is off unless you turn it on, and off means inert: no poller goroutine, no jobs key on the status endpoint, and a byte-identical ledger.

jobs:
  enabled: true

Everything else has a default worth knowing before you change one.

The default journal path is two jobs segments deep, which surprises people looking for it: a state directory named jobs beside the ledger, and jobs/jobs.jsonl inside that. The jobs plane armed line at boot prints the full path, which is the quickest way to confirm it.

When the settlement arrives

The schedule is fixed and knowable, which means the worst case is a number rather than a feeling. The first poll happens two seconds after the submit, with no jitter on it. Each interval after that doubles until it reaches the 60-second ceiling, and each one is multiplied by a jitter factor that is fixed for the life of the job and derived from its request id, landing somewhere in [0.8, 1.2]. Sixty attempts at those intervals sum to 2s + f × 3360s.

Plan for roughly 56 minutes, and treat 45 to 67 as the band. All three are lower bounds: the clock for the next interval starts when a poll finishes, so each attempt’s own round trip, up to 30 seconds of it, is added on top. jobs.poll.max_attempts and jobs.poll.backoff.max are the two knobs that move the number.

That horizon is also the bound on budget enforcement, which is worth planning for rather than discovering. Admission gates an async submit like any other request: scope, rate and concurrency limits all apply the moment the job is created. A spend gate is different. It sees the job’s cost only when the settlement row reaches the ledger, minutes to tens of minutes later.

No reservation against a predicted job cost is attempted at any point. A budget that turned a request away against an estimate of what an image generation might cost would be enforcing a figure nobody measured, so the exposure between submit and settlement is left open and disclosed rather than papered over with a guess. The disclosure is a field: every budget status carries pending_settlement_rows beside unpriced_rows, and a rising pending count beside a flat burn is the picture of a budget that has not yet been told about spend that has already happened. Alert on that field, not on burn alone.

One trap in the other direction: shortening max_attempts does not make the money arrive sooner. It converts a job that would have settled into one that is abandoned, which is a written outcome rather than a silent forgetting, but also a job whose cost you will not learn.

Three things this does not do

There is no synchronous facade

A wrapper that would have accepted an OpenAI-format /v1/images/generations, polled the provider internally, and answered with the finished image was planned, sequenced last, and dropped. What it would have sat on shipped in full: the journal, the poller, the settlement row, the seven providers. So your client still speaks the provider’s own async contract. Submit, take the provider’s own job id, poll the provider’s own status path through the gateway. We do not hide the wait for you.

There is no tokentriage jobs command

The job plane is observable in three places and no others. The jobs block on GET /tt/api/status appears when the plane is armed and is omitted when it is not. A jobs plane armed line at boot carries the journal path and how many pending jobs this process adopted from the last one. A jobs settlement summary line at shutdown carries what was carried forward, attempted, settled and abandoned. Pending jobs at shutdown are not a failure; they stay in the journal and are re-polled at the next boot.

The capabilities endpoint does not mention the poller

GET /tt/api/v1/capabilities carries no jobs, async, settlement or poller field at all. Its one egress-related field, judge_external_dependency, is computed from the evals evaluators and the shadow-duplication block and nothing else. A deployment with the job plane armed and neither of those configured reports judge_external_dependency: false while the poller is dialling a provider out of band.