Add a provider with your own key — TokenTriage Docs

Add an upstream to your config that authenticates to a provider with your own credential, where the config references the secret by its environment-variable name — so the secret value itself never lives in a file.

Prerequisites

A working tokentriage.yaml (see the quickstart).
The provider credential exported into your shell/orchestrator environment (e.g. export OPENAI_API_KEY=sk-…). TokenTriage reads the value at boot from the named variable.
Basic familiarity with the upstreams and providers model.

The credential model in one line

An upstream carries its BYO key one of two ways, selected by the provider family’s auth kind:

Auth kind	Provider families	Credential config
`bearer`	OpenAI-compatible (Groq, Together, vLLM, …)	`api_key_env:` — single env-var name
`header`	Anthropic, Azure OpenAI, Gemini (AI Studio)	`api_key_env:` — single env-var name
`aws_sigv4`	Amazon Bedrock	`auth:` block of env-var names (`access_key_id_env`, `secret_access_key_env`, …)
`gcp_oauth`	Google Vertex AI	`auth: { credentials_file_env: … }` — env var holding the SA-key file path

bearer and header both use the single api_key_env carrier; aws_sigv4 and gcp_oauth need the multi-part auth: block. The auth kind is supplied by the provider descriptor (a built-in one you name with provider:, or an inline one you declare) — you rarely set it by hand.

Step 1 — Add an OpenAI-compatible upstream (bearer)

Any upstream that speaks the OpenAI chat-completions wire names a built-in descriptor with provider: and points api_key_env: at your key’s env var. dialect is derived from the descriptor, so omit it.

upstreams:
  together:
    provider: together_ai              # built-in descriptor: openai wire, bearer auth
    base_url: https://api.together.xyz/v1
    api_key_env: TOGETHER_API_KEY      # env var NAME — never a secret literal

base_url is required even with provider: — descriptors carry only relative paths, no host.

Step 2 — Or an Anthropic upstream (header)

The Anthropic family uses a header auth scheme, but the credential is still carried by the single api_key_env field:

upstreams:
  anthropic:
    base_url: https://api.anthropic.com
    dialect: anthropic                 # openai | anthropic
    api_key_env: ANTHROPIC_API_KEY     # env var NAME

An upstream with no api_key_env forwards the inbound Authorization header as-is — useful for a local, keyless server:

upstreams:
  ollama:
    base_url: http://localhost:11434/v1
    dialect: openai
    # no api_key_env → inbound Authorization forwarded unchanged

Step 3 — Or a provider that needs more than a key

Two families carry credential or region facts a single api_key_env cannot.

Amazon Bedrock (aws_sigv4) signs every request, so it needs a region (which also selects the region-class price) and a multi-part credential block — each field an env-var name, never a literal:

upstreams:
  bedrock:
    provider: bedrock
    base_url: https://bedrock-runtime.us-east-1.amazonaws.com
    region: us-east-1                        # REQUIRED: signing + region-class pricing
    auth:
      access_key_id_env: AWS_ACCESS_KEY_ID
      secret_access_key_env: AWS_SECRET_ACCESS_KEY
      session_token_env: AWS_SESSION_TOKEN   # optional (STS / temporary creds)

A Bedrock bearer API key short-circuits SigV4: set api_key_env: alone instead of the auth: block.

Google Vertex AI ships as a built-in vertex_ai provider (Gemini wire dialect, gcp_oauth auth via a self-signed service-account JWT) — you don’t declare a descriptor. Reference it and supply your project and location as upstream params: (they fill the {project} / {location} slots in the descriptor’s URL template). The credential names an env var holding the SA-key file path:

upstreams:
  vertex:
    provider: vertex_ai                 # built-in descriptor (gemini dialect, gcp_oauth)
    base_url: https://us-central1-aiplatform.googleapis.com
    params:
      project: my-gcp-project           # fills the {project} URL slot
      location: us-central1             # fills the {location} URL slot
    auth:
      credentials_file_env: GOOGLE_APPLICATION_CREDENTIALS  # env var holding the SA-key file PATH

Step 4 — Validate strictly

tokentriage validate loads the config with strict decoding (an unknown key is an error) and prints a summary. It does not read the secret — only the reference — so you can validate before the env var is even set.

$ tokentriage validate --config tokentriage.yaml
mode:      shadow
listen:    127.0.0.1:8787   admin: 127.0.0.1:9090
upstreams: 2
  - openai                 https://api.openai.com/v1
  - together               https://api.together.xyz/v1
tiers:     2 (cheapest -> most capable)
  0. cheap      -> together/meta-llama/Llama-3.3-70B-Instruct-Turbo
  1. frontier   -> openai/gpt-4o
rules:     1
OK: configuration is valid

You should see OK: configuration is valid on the last line. A misreferenced credential or an unknown key fails here with a path-named error rather than at request time.

Step 5 — Confirm the model is priced under the right provider

Pricing binds to the provider, not the wire dialect — so a model at your new upstream prices under that provider’s catalog prefix. Look it up with the fully-qualified provider/model key:

$ tokentriage prices show together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo
model:      together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo
resolved:   together_ai/together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo
provider:   together_ai
snapshot:   litellm@2026-07-14#179affb
source:     litellm  fetched 2026-07-14  generator pricegen v2
upstream_sha256: 179affb29103fed8b02a83e22e93209c2c65a393cbf9dd785dbaec0f8748d6f7

RATE         USD / TOKEN          USD / 1M TOKENS
input        8.8e-07              $0.8800
output       8.8e-07              $0.8800
cache_read   (fallback = input)   $0.8800
cache_write  (fallback = input)   $0.8800
reasoning    (fallback = output)  $0.8800

The identical model string routed at an openai-prefix upstream would resolve unpriced (model_unknown) — a different, correct outcome — so a wrong-provider price is impossible by construction.

Why this is honest and safe

Secrets stay out of config. Only env-var names are written; the loader’s env-reference guard refuses anything resembling a raw value, so a credential can’t leak into a YAML file, a diff, or a config dump.
Strict decoding fails loud. A typo in a credential key or provider name is a validation error, not a silent fallback that would forward with the wrong (or no) credential.
Pricing is provider-bound. Your BYO upstream’s costs price under its own catalog prefix; a model the snapshot doesn’t know is honestly unpriced rather than assigned a confident wrong dollar.