Connect your app — TokenTriage Docs

Keep the provider client you already construct. Four edits route it through TokenTriage: its base URL, its credential, two request headers, and the model name in the body.

Your application holds one secret, a TokenTriage runtime key (ttv_…), and it travels as the bearer token on Authorization. It holds no provider key: your OpenAI credential is stored in TokenTriage as a saved connection and attached on the way out to the provider. That is what saved mode means, and it inverts the older managed shape, where Authorization belonged to the provider SDK and our key rode on X-Tokentriage-Key.

X-Tokentriage-Gateway names the gateway that routes the request, and it is required even when your credential allows exactly one. X-Tokentriage-Environment may be omitted only when the credential binds exactly one environment. "model": "gateway" is a sentinel rather than a model name: the gateway’s applied release chooses the provider model and substitutes it before dispatch, so the literal string reaches no provider.

Four values go in the shell that starts your backend: the assigned gateway base URL, a runtime key, the gateway ID (gw_…) and the environment ID (env_…).

$ export TOKENTRIAGE_GATEWAY_BASE='your-assigned-gateway-url'
$ export TOKENTRIAGE_RUNTIME_KEY='your-runtime-key'
$ export TOKENTRIAGE_GATEWAY_ID='your-gateway-id'
$ export TOKENTRIAGE_ENVIRONMENT_ID='your-environment-id'

To let the packaged client stamp those headers for you instead, read make your first routed request; to install that client from this checkout, see install and configure.

Python

Install the official provider package. Nothing else is needed on this path.

python -m pip install "openai==3.8.0"

This script sends one chat completion through the gateway and prints the answer, the request ID and the provider model the release chose.

import os
import sys

from openai import APIError, APIStatusError, OpenAI

NAMES = ("TOKENTRIAGE_GATEWAY_BASE", "TOKENTRIAGE_RUNTIME_KEY",
         "TOKENTRIAGE_GATEWAY_ID", "TOKENTRIAGE_ENVIRONMENT_ID")
missing = [name for name in NAMES if not os.environ.get(name)]
if missing:
    sys.exit(f"Set {', '.join(missing)} before running this example.")
base, runtime_key, gateway_id, environment_id = (os.environ[name] for name in NAMES)

try:
    with OpenAI(
        base_url=base.rstrip("/").removesuffix("/v1") + "/v1",
        api_key=runtime_key,
        default_headers={
            "X-Tokentriage-Gateway": gateway_id,
            "X-Tokentriage-Environment": environment_id,
        },
        max_retries=0,
    ) as client:
        response = client.chat.completions.with_raw_response.create(
            model="gateway",
            messages=[{"role": "user", "content": "Say hello in one sentence."}],
            max_tokens=128,
        )
        answer = response.parse().choices[0].message.content
except APIStatusError as error:
    # The gateway's refusal code is in the error body.
    sys.exit(f"Request failed: code={error.code} status={error.status_code} "
             f"request_id={error.response.headers.get('x-tt-request-id')} "
             f"transport_request_id={error.response.headers.get('x-request-id')}")
except APIError as error:
    # No response received: execution may have occurred. Do not retry blindly.
    sys.exit("Request outcome unknown: no response received; check Requests before retrying.")
except Exception as error:
    sys.exit(f"Request failed: {error!r}")

print(answer)
print("TokenTriage request ID:", response.headers.get("x-tt-request-id"))
print("Transport request ID:", response.headers.get("x-request-id"))
print("Model:", response.headers.get("x-tt-model"))

Expected output is three lines: the answer, then Request ID: with a req_-prefixed ID, then Model: with the provider model. A model name there rather than the string gateway is the evidence that routing happened. This snippet was run against a local fake gateway, which checked the credential and both selectors; it has not been run against the private deployment.

TypeScript

Install the official provider package, then run this in your backend, where the runtime key stays private.

npm install [email protected]

The TypeScript path sends the same request and reads the same three response headers.

import OpenAI from "openai";

const names = ["TOKENTRIAGE_GATEWAY_BASE", "TOKENTRIAGE_RUNTIME_KEY",
  "TOKENTRIAGE_GATEWAY_ID", "TOKENTRIAGE_ENVIRONMENT_ID"];
const missing = names.filter((name) => !process.env[name]);
if (missing.length > 0) {
  console.error(`Set ${missing.join(", ")} before running this example.`);
  process.exit(1);
}
const [base, runtimeKey, gatewayId, environmentId] = names.map((name) => process.env[name] ?? "");

try {
  // The OpenAI client holds no connection of its own to close: it uses the
  // runtime's shared fetch, which releases its sockets when the process exits.
  const client = new OpenAI({
    baseURL: base.replace(/\/+$/, "").replace(/\/v1$/, "") + "/v1",
    apiKey: runtimeKey,
    defaultHeaders: {
      "X-Tokentriage-Gateway": gatewayId,
      "X-Tokentriage-Environment": environmentId,
    },
    maxRetries: 0,
  });
  const { data, response } = await client.chat.completions.create({
    model: "gateway",
    messages: [{ role: "user", content: "Say hello in one sentence." }],
    max_tokens: 128,
  }).withResponse();
  console.log(data.choices[0]?.message.content);
  console.log("TokenTriage request ID:", response.headers.get("x-tt-request-id"));
  console.log("Transport request ID:", response.headers.get("x-request-id"));
  console.log("Model:", response.headers.get("x-tt-model"));
} catch (error) {
  if (error instanceof OpenAI.APIError) {
    // No response received does not establish whether execution occurred.
    console.error(`Request failed: code=${error.code ?? null} status=${error.status ?? null} `
      + `request_id=${error.headers?.get("x-tt-request-id") ?? null} `
      + `transport_request_id=${error.headers?.get("x-request-id") ?? null}`);
    if (!error.status) console.error("Outcome unknown; check Requests before retrying.");
  } else {
    console.error("Request failed:", error);
  }
  process.exitCode = 1;
}

x-tt-release, x-tt-gateway and x-tt-environment come back on the same response if you want to record which configuration served the call. This snippet was run against the same local fake and has not been run against the private deployment either.

When the request is refused instead

The likeliest mistake is leaving the provider key where it used to go. A saved-mode request that also carries a provider credential — on Authorization, or on a carrier such as X-Api-Key — is answered 401 provider_credential_not_allowed, because a request whose spending authority nobody can name is refused rather than routed. Remove the provider key from the client and let the saved connection supply it; the runtime key is the only credential this code holds.

Structured error metadata reads the code, status and request ID off the exception your provider client raises, and the first routed request tables the other named refusals with their fixes.

What this page does not cover

Streaming, async clients and keeping your own transport are on provider clients and streaming. Native Anthropic Messages is not a supported managed integration. Request-supplied mode, where a caller sends a provider key on X-Tokentriage-Provider-Key, is gateway configuration rather than a client choice, and the hosted product does not admit it today. A base-URL change on its own routes nothing: with no selector headers on the request, the gateway has nothing to route by and says so.