# Self-hosted and declared-free upstreams
Point at a model server you run, and get an explicit zero with provenance instead of a blank where a cost should be.
Source: https://tokentriage.com/docs/build/providers-self-hosted
---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
A model running on your own hardware costs you electricity and a GPU lease, not a per-token
invoice. So what should the gateway record for that request: zero, or nothing?
They are different answers, and conflating them is how a cost report ends up wrong. "Zero, because
this upstream charges nothing" is a figure you can add up and forward. "No figure, because I could
not find a rate" is a hole. TokenTriage keeps them apart, and 8 declared-free descriptors ship to
say the first one out loud. All 8 carry a gated cell in the generated provider matrix, which names
the test behind each and runs it against a mock rather than the provider's live endpoint.
## The config
Nothing but a base URL. Authentication is `none`, so there is no credential to name.
```yaml
upstreams:
ollama:
provider: ollama
base_url: http://127.0.0.1:11434 # host only — your client already sends /v1
tiers:
- name: local
upstream: ollama
model: llama3.1
```
All 8 share that shape, one per server you might be running:
| Descriptor | Server |
|---|---|
| `ollama` | Ollama |
| `vllm` | vLLM |
| `lmstudio` | LM Studio |
| `llamacpp` | llama.cpp |
| `tgi` | Hugging Face Text Generation Inference |
| `localai` | LocalAI |
| `lemonade` | AMD Lemonade Server |
| `local` | anything else OpenAI-shaped on your own network |
They differ only in the name, which is what the provenance stamp records. Pick the one that matches
what you are running; if nothing matches, `local` is the general case. TokenTriage ships 95 built-in
provider descriptors, all but ten of them carrying at least one gated cell. Two of the 95 are
generic shapes rather than vendors: `local`, and `openai_compatible`.
## What the row looks like
Send a request and the row carries a zero with a source, not a blank:
```json
"http": { "endpoint": "chat", "provider": "ollama", "status": 200 },
"usage": { "input_tokens": 11, "output_tokens": 5, "total_tokens": 16,
"source": "provider" },
"cost": { "usd": 0, "pricing_key": "",
"pricing_snapshot": "descriptor@ollama#78964d3",
"estimated": false, "would_usd": null }
```
That capture is from a local OpenAI-shaped mock, so the token counts are the mock's; the zero and
the stamp are the descriptor's assertion and do not depend on them.
Four fields say what happened. `usd` is zero. `pricing_key` is empty, because no catalog lookup
took place. `pricing_snapshot` is a `descriptor@` stamp rather than a price-table snapshot, which
is how you tell an asserted zero from a resolved rate that happens to be zero. `would_usd` is
`null` because there was no counterfactual tier to reprice against.
There is no `unpriced` flag on that row. A declared-free zero is a priced fact, so it stays inside
cost totals and outside unpriced triage. An unpriced row is the opposite: it carries
`"unpriced": true` and a typed reason, and it is excluded from totals with the exclusion stated.
Usage is still recorded in full. Tokens, latency, routing decision, cache outcome and everything
else land on the row as they would for a paid upstream, so a local tier still shows up in your
traffic and quality reporting. Only the dollar is asserted.
`provider:` supplies the wire dialect and the endpoint list even when there is no price to look up
and no credential to inject. It is also what the provenance stamp is built from, so
`descriptor@ollama#…` and `descriptor@vllm#…` are distinguishable in a ledger months later.
The seven hex characters are the first of a SHA-256 over the string `descriptor-free:`, so
the stamp is a stable identity for the descriptor that asserted the zero rather than a content hash
of the descriptor itself. The pricing engine cannot read descriptor bytes, which is why the
identity is the name. `descriptor@ollama#78964d3` is the same string on every build.
## Why you would put a local model behind the gateway at all
**To compare it against a paid tier.** A local model as the cheap tier and a hosted model as the
frontier gives shadow mode something to grade, and the counterfactual it computes is "what would
this have cost on the paid tier", which is the number the local hardware has to beat.
**To keep some traffic off the network entirely.** Classification, extraction and reformatting
often do not need a frontier model, and routing them locally is a spend reduction that shows up as
paid rows that stopped happening.
**To keep the cost surface complete.** Rows with no dollar at all would make every total a partial
one. An asserted zero keeps the arithmetic whole.
`ollama` and `lemonade` declare no catalog namespace, and their names happen to match a prefix in
the vendored `litellm@2026-07-14#179affb` price file. The coverage generator counts a declared-free
descriptor whose name matches an artifact prefix, so its identity line reads 83 prefix-bound plus 2
name-matched declared-free = 85 covered prefixes, of the 107 in that artifact, leaving 22
uncovered. `openai_compatible` is not in that sum: it binds a catalog prefix that is not one of the
107, so the generator counts it as covering nothing.
Covered is not served, and the generator prints that too: `oci` is inside the 85 and reaches 19 of
its 44 priced keys, with the 25 it cannot reach published beside them. The catalog's own rows under
`ollama/` and `lemonade/` are not read either, because a declared-free upstream resolves no key at
all.
## Asserting free for an upstream that is not self-hosted
A committed-capacity deal, an internal chargeback arrangement, a provider you pay a flat fee: the
zero is yours to assert, and a descriptor is what carries the assertion. Declare one in config
rather than picking a built-in that names somebody else's server.
```yaml
providers:
reserved-capacity: # the descriptor id you name below
dialect: openai
pricing: { free: true } # your negotiated reality, asserted
upstreams:
reserved-capacity:
provider: reserved-capacity
base_url: https://inference.internal.example.com
api_key_env: INTERNAL_INFERENCE_KEY
```
An inline descriptor's `pricing:` takes either `free: true` or a `catalog_prefix`, and one of the
two is required. `free: true` binds no namespace, so nothing is looked up and every row through
that upstream carries `usd: 0` stamped `descriptor@reserved-capacity#…`, the same shape the 8
built-ins produce. Validation accepts the assertion without argument, because the arrangement is
yours to know. The rest of the inline-descriptor form, including endpoints, auth and per-endpoint
paths, is on [the custom-provider page](/docs/build/providers-custom).
## What can go wrong
| Symptom | Cause | Fix |
|---|---|---|
| connection refused | the server is not running, or is on another port | check the base URL; Ollama's OpenAI surface is under `/v1` |
| rows show a dollar you did not expect | the descriptor is a paid one, not a declared-free one | use one of the 8, or declare an inline descriptor with `pricing: { free: true }` |
| `providers["…"].dialect: required` | an inline descriptor with no `dialect:` | inline descriptors are OpenAI-wire refinements; set `dialect: openai` |
| `pricing.catalog_prefix is required unless the provider is declared-free` | an inline descriptor with neither `free: true` nor a prefix | pick one; a descriptor with no pricing binding is refused |
| `pricing_key` empty and `usd` zero on every row | a declared-free upstream, working as configured | nothing to fix; check the stamp names the descriptor you meant |
| totals look too low | a local tier is carrying more traffic than you thought | that is the saving; check the routing decision counts |