SDKs
Point the OpenAI Node SDK at TokenTriage
The /v1 is yours to write.
One option changes. Point baseURL at the root plus /v1, and the ledger gets a dialect, a model and a price.
import OpenAI from 'openai';
// Root + /v1. The SDK appends /chat/completions and repairs nothing.
const client = new OpenAI({
apiKey: process.env.OPENAI_API_KEY,
baseURL: 'http://127.0.0.1:8787/v1',
});
// Or export OPENAI_BASE_URL=http://127.0.0.1:8787/v1 and drop the option.
const res = await client.chat.completions.create({
model: 'gpt-4o-mini',
messages: [{ role: 'user', content: 'hello' }],
});- The setting
baseURL — a new OpenAI() option, or OPENAI_BASE_URL- The base URL
http://127.0.0.1:8787/v1 — OPENAI_BASE_URL takes the /v1 too
How to tell it worked
Read the row, not the response. A costed call stamps http.dialect openai and a pricing key. A passthrough leaves both empty, whatever status your client saw.
import { readFileSync } from 'node:fs';
import { homedir } from 'node:os';
const log = `${homedir()}/.tokentriage/decisions.jsonl`;
const row = JSON.parse(readFileSync(log, 'utf8').trim().split('\n').at(-1));
console.log(row.http.path, row.http.dialect || '(none)', row.cost.pricing_key || '(none)');What you should see
/v1/chat/completions openai openai/gpt-4o-mini · uncosted prints /health (none) (none)
How sure we are
Reproduced
A person ran it by hand against a real gateway on a stated date, and the transcript is on the page.
Verified against openai 7.5.0 on Node v24.13.0 on
node_version "v24.13.0"
platform "win32 x64"
openai_npm_version "7.5.0"
openai_npm_engines {"node":">=22.0.0"}
openai_npm_runtime_dependencies []
harness_stderr "ttharness: up (mode=route tier=fast refusal=false unknown-model=false fault=false). Close stdin or SIGINT to stop.\nttharness: shutting down"
[A] root + /v1, non-streaming · LAB RUN · MOCK UPSTREAM
Q: Does the openai npm package, at baseURL = <root>/v1, produce a COSTED ledger row?
CLIENT CONFIG {"baseURL":"http://127.0.0.1:12683/v1","call":"chat.completions.create"}
WIRE POST http://127.0.0.1:12683/v1/chat/completions -> 200
LEDGER {"req_id":"tt_77a54nccwupjo6fsuds23xnhyu","ts":"2026-08-25T20:58:15.736Z","http.path":"/v1/chat/completions","http.dialect":"openai","http.stream":false,"http.status":200,"request.model":"gpt-4o-mini","decision.mode":"route","decision.tier":"fast","usage.total_tokens":120,"usage.source":"provider","cost.usd":0.000027,"cost.pricing_key":"openai/gpt-4o-mini","cost.estimated":false}
[F] bare root, GET /health through the SDK · LAB RUN · MOCK UPSTREAM
Q: An unmatched path the upstream DOES serve: does it 200, and is the row costed?
CLIENT CONFIG {"baseURL":"http://127.0.0.1:12683","call":"client.get('/health')"}
WIRE GET http://127.0.0.1:12683/health -> 200
CLIENT OK {"status":200,"body":"OK"}
LEDGER {"req_id":"tt_hslxqcbpykj66cagqzm32k4gyu","ts":"2026-08-25T20:58:15.773Z","http.path":"/health","http.dialect":"","http.stream":false,"http.status":200,"request.model":"","decision.mode":"route","decision.tier":"","usage.total_tokens":0,"usage.source":"absent","cost.usd":0,"cost.pricing_key":"","cost.estimated":true}The gotcha
A bare root can answer 200 and cost nothing
The status is not the test. The SDK’s own call 404’d here, because the mock served two routes. A GET to /health on the same bare root returned 200 and body OK. The row for it carried no dialect, no model and no price.
const base = process.env.OPENAI_BASE_URL ?? 'http://127.0.0.1:8787/v1';
if (!new URL(base).pathname.replace(/\/$/, '').endsWith('/v1')) {
throw new Error(`baseURL ${base} has no /v1 — chat calls will pass through uncosted`);
}
const client = new OpenAI({ baseURL: base, apiKey: process.env.OPENAI_API_KEY });“@param {string} [opts.baseURL=process.env['OPENAI_BASE_URL'] ?? https://api.openai.com/v1] - Override the default base URL for the API.”
openai 7.5.0 · src/client.ts:448, from the shipped tarball · read 2026-08-25
What you get now
What the row carries once one service points here.
Three things a Node service gets the moment its first call lands. Every item links its evidence. None of it needs a code change.
A row per call, with the price on it
TokenTriage parses the OpenAI dialect, reads the provider’s own usage numbers and prices them from a dated snapshot. The price is not a guess. This run’s costed rows carried cost.estimated false.
What a ledger row holdsStreamed calls carry real token counts
TokenTriage asks the upstream for the usage frame and takes it back out of your copy. You do not opt in. In this run a plain stream showed eight chunks, no usage frame, and a priced row.
Where the usage frame comes fromA cap that lives with the key, not the code
TokenTriage mints keys here and stores them as hashes, with budgets and rate limits attached to tenancy nodes. The cap follows the key. A runaway loop in one service spends against its own node.
How budgets and keys work
- 1 option changed
- baseURL, or OPENAI_BASE_URL — the client is otherwise untouched
- 0 runtime dependencies
- openai 7.5.0 declares none, and node_modules holds one directory
- 10 scenarios
- each one correlated to the ledger rows it produced, on 2026-08-25
- 2 paths that answered 200
- one carried a pricing key, the other carried no dialect at all
Where this stops
Six things to know before you cut a service over.
Three belong to the gateway, two to the SDK, one to our evidence. The last one is about the chip. Read that one first.
Off the registered path, nothing is parsed
Parsing, pricing and endpoint-scoped governance run on a registered path. Elsewhere the call is relayed with the path as written, and that lane decides streaming by scanning raw bytes. The chat path parses the JSON.
Which address a client should getThe path is exact, and you can write it
A trailing slash is a different path. client.post('/chat/completions/') 404’d, and its row carried no dialect. chat.completions.create never writes that, even from a base ending in a slash — the SDK de-duplicates it.
Your bytes are not the bytes upstream sees
On a streamed chat call TokenTriage splices stream_options.include_usage into the outbound body and removes the resulting frame from your copy. Your chunks do not change. A strict upstream, or anything diffing the body, sees the difference.
The splice, and the switch that turns it offNode 22 is the package’s own floor
openai 7.5.0 declares engines.node >= 22.0.0 and ships zero runtime dependencies. A reader on Node 20 is outside that support before TokenTriage enters. This run used Node v24.13.0 only.
Per-tenant keys stop at the gateway
Where an upstream names api_key_env, TokenTriage strips the client’s credential and replaces it. One service multiplexing several tenants’ keys through one base URL loses that distinction. Mint a key per tenant instead.
A key per tenant, with a capThe chip covers a run, not a gate
No test in the product repo drives the openai npm package against TokenTriage. A person ran ten scenarios on a stated date, through the real proxy engine in front of a mock upstream. No provider was called.
Change one option, then read one row.
Move a single service first. Send one call and read the row. Move the next when the pricing key shows up every time.
Names and trademarks
These are independent integrations. No product named here has endorsed, reviewed or sponsored TokenTriage; every name is its owner's trademark, used only to say what works with what.