Self-hosted and declared-free upstreams

A model running on your own hardware costs you electricity and a GPU lease, not a per-token invoice. So what should the gateway record for that request: zero, or nothing?

They are different answers, and conflating them is how a cost report ends up wrong. “Zero, because this upstream charges nothing” is a figure you can add up and forward. “No figure, because I could not find a rate” is a hole. TokenTriage keeps them apart, and 8 declared-free descriptors ship to say the first one out loud. All 8 carry a gated cell in the generated provider matrix, which names the test behind each and runs it against a mock rather than the provider’s live endpoint.

The config

Nothing but a base URL. Authentication is none, so there is no credential to name.

upstreams:
  ollama:
    provider: ollama
    base_url: http://127.0.0.1:11434   # host only — your client already sends /v1

tiers:
  - name: local
    upstream: ollama
    model: llama3.1

All 8 share that shape, one per server you might be running:

Descriptor	Server
ollama	Ollama
vllm	vLLM
lmstudio	LM Studio
llamacpp	llama.cpp
tgi	Hugging Face Text Generation Inference
localai	LocalAI
lemonade	AMD Lemonade Server
local	anything else OpenAI-shaped on your own network

They differ only in the name, which is what the provenance stamp records. Pick the one that matches what you are running; if nothing matches, local is the general case. TokenTriage ships 95 built-in provider descriptors, all but ten of them carrying at least one gated cell. Two of the 95 are generic shapes rather than vendors: local, and openai_compatible.

What the row looks like

Send a request and the row carries a zero with a source, not a blank:

"http":  { "endpoint": "chat", "provider": "ollama", "status": 200 },
"usage": { "input_tokens": 11, "output_tokens": 5, "total_tokens": 16,
           "source": "provider" },
"cost":  { "usd": 0, "pricing_key": "",
           "pricing_snapshot": "descriptor@ollama#78964d3",
           "estimated": false, "would_usd": null }

That capture is from a local OpenAI-shaped mock, so the token counts are the mock’s; the zero and the stamp are the descriptor’s assertion and do not depend on them.

Four fields say what happened. usd is zero. pricing_key is empty, because no catalog lookup took place. pricing_snapshot is a descriptor@ stamp rather than a price-table snapshot, which is how you tell an asserted zero from a resolved rate that happens to be zero. would_usd is null because there was no counterfactual tier to reprice against.

Usage is still recorded in full. Tokens, latency, routing decision, cache outcome and everything else land on the row as they would for a paid upstream, so a local tier still shows up in your traffic and quality reporting. Only the dollar is asserted.

Why a declared-free upstream still needs a descriptor name

provider: supplies the wire dialect and the endpoint list even when there is no price to look up and no credential to inject. It is also what the provenance stamp is built from, so descriptor@ollama#… and descriptor@vllm#… are distinguishable in a ledger months later.

The seven hex characters are the first of a SHA-256 over the string descriptor-free:<name>, so the stamp is a stable identity for the descriptor that asserted the zero rather than a content hash of the descriptor itself. The pricing engine cannot read descriptor bytes, which is why the identity is the name. descriptor@ollama#78964d3 is the same string on every build.

Why you would put a local model behind the gateway at all

To compare it against a paid tier. A local model as the cheap tier and a hosted model as the frontier gives shadow mode something to grade, and the counterfactual it computes is “what would this have cost on the paid tier”, which is the number the local hardware has to beat.

To keep some traffic off the network entirely. Classification, extraction and reformatting often do not need a frontier model, and routing them locally is a spend reduction that shows up as paid rows that stopped happening.

To keep the cost surface complete. Rows with no dollar at all would make every total a partial one. An asserted zero keeps the arithmetic whole.

Why two of the 8 also count as covered catalog prefixes

ollama and lemonade declare no catalog namespace, and their names happen to match a prefix in the vendored litellm@2026-07-14#179affb price file. The coverage generator counts a declared-free descriptor whose name matches an artifact prefix, so its identity line reads 83 prefix-bound plus 2 name-matched declared-free = 85 covered prefixes, of the 107 in that artifact, leaving 22 uncovered. openai_compatible is not in that sum: it binds a catalog prefix that is not one of the 107, so the generator counts it as covering nothing.

Covered is not served, and the generator prints that too: oci is inside the 85 and reaches 19 of its 44 priced keys, with the 25 it cannot reach published beside them. The catalog’s own rows under ollama/ and lemonade/ are not read either, because a declared-free upstream resolves no key at all.

Asserting free for an upstream that is not self-hosted

A committed-capacity deal, an internal chargeback arrangement, a provider you pay a flat fee: the zero is yours to assert, and a descriptor is what carries the assertion. Declare one in config rather than picking a built-in that names somebody else’s server.

providers:
  reserved-capacity:            # the descriptor id you name below
    dialect: openai
    pricing: { free: true }     # your negotiated reality, asserted

upstreams:
  reserved-capacity:
    provider: reserved-capacity
    base_url: https://inference.internal.example.com
    api_key_env: INTERNAL_INFERENCE_KEY

An inline descriptor’s pricing: takes either free: true or a catalog_prefix, and one of the two is required. free: true binds no namespace, so nothing is looked up and every row through that upstream carries usd: 0 stamped descriptor@reserved-capacity#…, the same shape the 8 built-ins produce. Validation accepts the assertion without argument, because the arrangement is yours to know. The rest of the inline-descriptor form, including endpoints, auth and per-endpoint paths, is on the custom-provider page.

What can go wrong

Symptom	Cause	Fix
connection refused	the server is not running, or is on another port	check the base URL; Ollama’s OpenAI surface is under /v1
rows show a dollar you did not expect	the descriptor is a paid one, not a declared-free one	use one of the 8, or declare an inline descriptor with pricing: { free: true }
providers["…"].dialect: required	an inline descriptor with no dialect:	inline descriptors are OpenAI-wire refinements; set dialect: openai
pricing.catalog_prefix is required unless the provider is declared-free	an inline descriptor with neither free: true nor a prefix	pick one; a descriptor with no pricing binding is refused
pricing_key empty and usd zero on every row	a declared-free upstream, working as configured	nothing to fix; check the stamp names the descriptor you meant
totals look too low	a local tier is carrying more traffic than you thought	that is the saving; check the routing decision counts