Providers

TokenTriage speaks a few wire dialects in code and carries 80+ providers as data — one built-in descriptor per provider. Adding an OpenAI-compatible provider is therefore an N+1-by-config change, with no new Go code.

Wire dialects

The shipped binary wires seven wire-dialect drivers: openai, anthropic, bedrock, gemini, anthropic_vertex (Claude-on-Vertex), elevenlabs (text-to-speech), and deepgram (speech-to-text). openai and anthropic are also the two client-facing dialects the proxy accepts (/v1/chat/completions and /v1/messages); the rest are upstream/egress dialects. Every OpenAI-compatible upstream rides the openai driver, so it needs zero wire code — only a descriptor and an upstream that names it.

The support matrix

81 descriptors ship built-in, and every one is real-run-verified: a mock upstream shaped like that provider is driven through the proxy in the e2e suite with byte-fidelity, cost, and honesty assertions — nothing is listed as “supported” that the test suite can’t confirm. A representative slice (the full per-provider matrix lives in PROVIDER_MATRIX.md):

Provider (provider:)	Wire	Auth	Pricing prefix	Proven end-to-end
openai	openai	bearer	openai/	chat + responses, embeddings, images, audio, models
anthropic	anthropic	header (x-api-key)	anthropic/	chat (/v1/messages) stream + non-stream
azure_openai	openai	header (api-key)	azure/	chat + embeddings; deployment-path + api-version
bedrock	bedrock	AWS SigV4	bedrock/ (region-in-key)	chat; SigV4 + CRC binary event-stream + region pricing
bedrock_converse	bedrock	AWS SigV4	bedrock_converse/ (region-in-key)	chat; same Converse wire, separate price bucket (138 models)
gemini	gemini	header (x-goog-api-key)	gemini/	chat; alt=sse streaming; AI-Studio + Vertex
vertex_ai	gemini	gcp_oauth (SA JWT)	vertex_ai/	chat; Gemini-on-Vertex, projects/locations URL
vertex_ai_anthropic	anthropic_vertex	gcp_oauth (SA JWT)	vertex_ai-anthropic_models/	chat; Claude-on-Vertex path-borne :rawPredict/:streamRawPredict
elevenlabs	elevenlabs	header (xi-api-key)	elevenlabs/ (per-character)	text-to-speech; per-char billed from the request text
deepgram	deepgram	header (Authorization: Token)	deepgram/ (per-second)	speech-to-text; per-second from the response duration
recraft	openai	bearer	recraft/ (per-image)	image generation (/v1/images/generations)
groq, together_ai, mistral, deepseek, xai, … (45 swept)	openai	bearer	own prefix	chat; each real-run-routed + DP2-priced
local, ollama, vllm, lmstudio, … (7 self-hosted)	openai	none	(declared-free $0)	chat; explicit $0, not an unpriced miss
openai_compatible	openai	bearer	(operator sets)	the generic escape hatch for any OpenAI-shaped upstream

Endpoint coverage spans chat, responses, embeddings, rerank, images, and audio (TTS + STT) — priced per-token, per-image, per-character, or per-second as the catalog dictates. The 45 swept OpenAI-compatible providers (Fireworks, OpenRouter, Perplexity, xAI, Mistral, DeepSeek, Cohere, and more) are built-in descriptors — provider: <name> with no Go code — as are the 7 self-hosted profiles (vLLM, Ollama, LM Studio, llama.cpp, TGI, LocalAI, Lemonade), Gemini- and Claude-on-Vertex, and the 9 Vertex Model Garden families. openai_compatible remains the escape hatch for any upstream not yet in the catalog.

localhost:9090/ui/providersDemo data

Add an upstream

Adding a provider is a few config lines — dialect is derived from the descriptor, and api_key_env is an environment-variable name, never a secret literal:

upstreams:
  together:
    provider: together_ai            # built-in descriptor: openai wire, bearer, together_ai/ pricing
    base_url: https://api.together.xyz/v1
    api_key_env: TOGETHER_API_KEY    # an env var NAME — never a secret literal

Families that need more than a base URL:

• Azure (provider: azure_openai): add params: { deployment, api_version } (a closed, validated set).
• Bedrock (provider: bedrock): add region: (required — for SigV4 signing and region-class pricing) and an auth: block (access_key_id_env, secret_access_key_env, optional session_token_env) — or an api_key_env bearer short-circuit.
• Vertex (provider: vertex_ai for Gemini, vertex_ai_anthropic for Claude — both built-in): add params: { project, location } and an auth: block with a credentials_file_env (the env var holds the service-account key file path, gcp_oauth).
• Audio (provider: elevenlabs TTS, deepgram STT — built-in): a base_url and an api_key_env; billed per-character / per-second from the request or response.
• local / declared-free (provider: local, ollama, vllm, …): just a base_url, auth none.

Cross-dialect translation

Opt-in cross-dialect translation (routing.allow_cross_dialect, default off) covers the openai ↔ anthropic pair and discloses every parameter it drops rather than silently swallowing it. Two edges are deferred: a streaming cross-dialect route falls back to the same-dialect upstream (no SSE cross-translation), and a non-2xx upstream reply is relayed verbatim in the served dialect’s error shape.

The full operator guide — per-provider config, the N+1 recipe, multi-part credentials, and the translation opt-in — is docs/PROVIDERS.md. See also the Configuration reference.