# Connect an OpenAI-compatible provider

Point TokenTriage at a provider in six lines of config, check it three ways before you send traffic, and confirm the first request priced correctly.

Source: https://tokentriage.com/docs/build/providers-connect

---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';

Any upstream that speaks the OpenAI chat-completions wire connects the same way, whether it is
Groq, Together, Fireworks, OpenRouter, DeepSeek, a Cerebras endpoint or a vLLM server on your own
hardware. This page walks that path once, with Groq, and ends with a request that comes back
carrying the price it was looked up at.

You need the provider's base URL and an API key. You do not need a database, an account with us,
or any change to your application code beyond its base URL.

This is the self-hosted path, where your config names a variable and the process reads its value at
startup, so nothing in the file has to be rotated. A hosted gateway has a second mode: the provider
key is saved once as a [provider connection](/docs/build/provider-connections), stored encrypted at
rest under a customer-managed KMS key in TokenTriage's cloud account, and not readable back through
any API, export or log. Request-supplied mode is the third case and it keeps the no-storage
guarantee, one request at a time. The mode belongs to the gateway's configuration and not to a
caller.

## 1. Name the upstream

An upstream is a block under `upstreams:` with a name you choose. `provider:` names a built-in
descriptor, which already carries the wire dialect, the authentication scheme and the pricing
namespace, so you do not repeat any of them.

```yaml
upstreams:
  groq:
    provider: groq                 # built-in descriptor: openai wire, bearer auth
    base_url: https://api.groq.com/openai   # /openai is Groq's — your client already sends /v1
    api_key_env: GROQ_API_KEY      # an env var NAME, never a secret literal
```

`api_key_env` holds the *name* of an environment variable. Pasting the key itself is the mistake
this field exists to prevent, and [the credential model](/docs/build/providers-credentials)
explains what the validator does and does not catch.

`dialect:` is derived from the descriptor, so leave it out. One exception is worth knowing before
you meet it: an upstream that a cross-dialect tier will target has to name `dialect:` alongside
`provider:`, and [cross-dialect translation](/docs/build/providers-cross-dialect) shows both
captures. `base_url` is always required, because a descriptor carries relative paths and no host.

Those five lines under a `version: 1` are already a config that loads. Six lines in total, and
`validate` accepts them on their own, in ledger mode, on the default listen and admin addresses.
Step 2 adds the tier that gives routing something to choose.

<Details summary="Where do I find my provider's descriptor name?">

The `provider:` value is the descriptor id, which is usually the provider's own catalog prefix:
`groq`, `together_ai`, `fireworks_ai`, `deepseek`, `mistral`, `openrouter`, `xai`, `cerebras`.
The complete list, with the pricing prefix and endpoints each one serves, is in
[the provider reference](/docs/build/providers-reference). A name that is not in the registry
fails validation and prints what you typed.

</Details>

## 2. Point a tier at it

Routing needs at least one tier, and a tier is an upstream plus a model string. This config is
otherwise minimal: shadow mode, one tier, no rules.

```yaml
version: 1

server:
  listen: "127.0.0.1:8787"
  admin:  "127.0.0.1:9090"

mode: shadow

upstreams:
  groq:
    provider: groq
    base_url: https://api.groq.com/openai
    api_key_env: GROQ_API_KEY

tiers:
  - name: cheap
    upstream: groq
    model: llama-3.3-70b-versatile

routing:
  default_tier: cheap
```

## 3. Validate it

`validate` loads the file, compiles the provider registry the way boot does, and prints what it
built. It dials nothing and reads no credential.

```console
$ tokentriage validate --config tokentriage.yaml
mode:      shadow
listen:    127.0.0.1:8787   admin: 127.0.0.1:9090
upstreams: 1
  - groq                   https://api.groq.com/openai
tiers:     1 (cheapest -> most capable)
  0. cheap      -> groq/llama-3.3-70b-versatile
rules:     0
OK: configuration is valid
```

The blank column between the upstream name and the URL is where the wire dialect prints. It is
empty for every upstream configured with `provider:`, because the derived dialect is not written
back into the config the summary reads. That is a display gap, not a misconfiguration; the
descriptor's dialect is in force either way, and
[the dialect matrix](/docs/build/providers-dialect-matrix) is where you read which client formats
reach which upstreams.

## 4. Check the credential is actually set

Validation checks that `api_key_env` is a legal variable name. It cannot check that the variable
exists, because that depends on the shell you eventually start the binary in. `doctor` can.

```console
$ tokentriage doctor --config tokentriage.yaml
✓ binary — tokentriage dev (none), embedded pricing snapshot litellm@2026-07-14#179affb — clean
✓ config — tokentriage.yaml — strict decode clean: mode shadow, 1 upstream(s), 1 tier(s), 0 rule(s)
✓ compile — provider registry + routing policy compiled — 1 upstream(s) wired — clean
✓ ports — listen 127.0.0.1:8787 bindable, admin 127.0.0.1:9090 bindable — clean
✓ pricing — embedded table litellm@2026-07-14#179affb loaded, no overrides configured — clean
! keys — 1 upstream(s), 1 *_env reference(s): 1 unset (GROQ_API_KEY) — missing_env
    set: export GROQ_API_KEY=…   (referenced by an upstream but unset)

ok — no hard failures, 1 soft warning(s) (exit 0)
```

That capture is from a source build, so it prints `dev` where a release prints a version, and the
config path is this machine's. Five further check lines are cut for length: `ledger`, `index`,
`dashboard`, `instance` and `features`. The `keys` line is the one to read: it reports presence by
variable *name*, and never the value.

Export the key and the line turns green. An unset key is a soft warning rather than a failure
because forwarding whatever credential the client sent is a supported way to run.

## 5. Send one request and read the price

Point a client at the proxy and send a normal OpenAI request.

```console
$ curl -s http://127.0.0.1:8787/v1/chat/completions \
    -H 'content-type: application/json' \
    -d '{"model":"llama-3.3-70b-versatile",
         "messages":[{"role":"user","content":"hello"}]}'
```

Then read the row it wrote. The field to check is `cost.pricing_key`: it should start with your
provider's namespace, not the wire dialect's.

```json
"http":  { "path": "/v1/chat/completions", "dialect": "openai", "status": 200,
           "endpoint": "chat", "provider": "groq" },
"usage": { "input_tokens": 11, "output_tokens": 5, "total_tokens": 16,
           "source": "provider" },
"cost":  { "usd": 1.044e-05, "pricing_key": "groq/groq/llama-3.3-70b-versatile",
           "pricing_snapshot": "litellm@2026-07-14#179affb", "estimated": false }
```

That row came from a mock upstream shaped like Groq, so the token counts are the mock's and the
dollar is arithmetic over the committed rate table rather than anyone's bill: 11 input tokens at
$0.59 per million plus 5 output tokens at $0.79 per million is $0.00001044. Against the live
provider the shape is identical and the counts are theirs.

The doubled `groq/` is not a bug. The vendored catalog keys many providers as
`<prefix>/<the model id the provider publishes>`, and that published id already carries the
prefix. [The pricing-key page](/docs/build/providers-pricing-keys) shows the resolution order
that produces it.

<Callout type="tip">
A third check runs before any traffic moves: `tokentriage prices show groq/llama-3.3-70b-versatile`
prints the resolved key, the snapshot it came from and the per-token rates, and dials nothing. Pass
the prefix; a bare model string resolves across all providers and can show you a different vendor's
number.
</Callout>

## What can go wrong

| Symptom | Cause | Fix |
|---|---|---|
| `does not name a known provider` | descriptor id misspelled | check the id in [the reference](/docs/build/providers-reference) |
| `base_url: must not be empty` | `provider:` set without a host | descriptors carry relative paths only |
| `this looks like a secret value` | a key literal in `api_key_env` | put the variable name there instead |
| row has `unpriced: true` | the model has no rate under this prefix | [diagnose the key](/docs/build/providers-pricing-keys) |
| provider returns 401 | the referenced variable is unset | run `doctor`; the `keys` check names it |

The 401 case is worth pre-empting. With `api_key_env` unset, requests are forwarded with whatever
credential the client sent, so a client sending nothing gets the provider's own 401 rather than an
error from the gateway. The boot banner says so at startup for any `api_key_env` it cannot resolve.

<NextSteps items={[
  {title: "Which pricing key your model lands under", href: "/docs/build/providers-pricing-keys", description: "Predict the key before you send anything, and tell a catalog gap from a job whose money is not known yet."},
  {title: "How credentials are configured", href: "/docs/build/providers-credentials", description: "You named one environment variable above. Five other fields take one, and a pasted key gets past more of them than you would expect."},
  {title: "Provider reference", href: "/docs/build/providers-reference", description: "You picked a name for provider: above. This is where you check that the one you want exists, and what else it will ask you for."},
  {title: "Add a provider TokenTriage does not ship", href: "/docs/build/providers-custom", description: "The generic OpenAI-shaped base, and declaring your own descriptor in config."}
]} />