Make your first routed request

In this tutorial we’ll send one chat completion through an assigned TokenTriage gateway and print the answer, the request ID, and the provider model the gateway picked for it.

Your application carries one credential: a TokenTriage runtime key, the ttv_-prefixed secret minted for a service identity. It carries no provider key. Your OpenAI or Anthropic credential is stored in TokenTriage as a saved connection and attached on the way out to the provider, by a process your application does not talk to. That is what saved mode means, and it is the only credential mode the hosted product admits today.

A runtime key opens one door: inference through the gateways its allowlist names. It cannot read the dashboard and it cannot call the management API. Those take a portal session or a management key (ttm_…), which is a different credential with a different lifetime.

What you need

Thing	Where it comes from
runtime key (`ttv_…`)	`tokentriage cloud credential create`, printed once
assigned gateway base URL	the application’s gateway page
gateway ID (`gw_…`)	the same page; the credential’s allowlist names it
environment ID (`env_…`)	the workspace’s environments; the credential binds one
the local SDK build	install the local build

You do not need a provider API key, a config file, a database, or dashboard access. The saved provider connection and the credential are set up once by whoever holds connections.manage and keys.manage; adding a saved provider connection is that side of the work.

About two minutes once you hold those four values. Minting the credential and saving the provider connection are not counted; they happen before this page.

1. Know the shape you are about to send

Four things carry a saved-mode request, and one of them is the body:

POST <assigned-gateway-base>/v1/chat/completions
Authorization: Bearer <TokenTriage runtime key>
X-Tokentriage-Gateway: gw_…
X-Tokentriage-Environment: env_…

{"model": "gateway", …}

That block describes the SDK’s request contract. The executable examples below are checked over real loopback HTTP with synthetic responses; live acceptance separately exercises the packaged applications against saved connections. These are different layers of evidence.

Read it line by line, because every line inverts something you may have done before. Authorization is ours in this mode: it holds the runtime key, because there is no provider key to put there. X-Tokentriage-Gateway is required even when the credential allows one gateway, so that every example and every runbook reads the same. X-Tokentriage-Environment may be omitted only when the credential binds one environment, and a value you do supply has to be that one.

"model": "gateway" is a sentinel, not a model name. The gateway’s applied release decides which provider model serves the request and substitutes it before dispatch, so the literal string gateway reaches no provider. Naming a provider model yourself is refused rather than quietly ignored.

2. Send it from Python

The packaged client stamps all three headers for you. connection.openai() takes the two selectors and no provider-key argument at all.

import os
import sys

from openai import APIError
from tokentriage.client import TokenTriage, TokenTriageError, error_details

NAMES = ("TOKENTRIAGE_GATEWAY_BASE", "TOKENTRIAGE_RUNTIME_KEY",
         "TOKENTRIAGE_GATEWAY_ID", "TOKENTRIAGE_ENVIRONMENT_ID")
missing = [name for name in NAMES if not os.environ.get(name)]
if missing:
    sys.exit(f"Set {', '.join(missing)} before running this example.")
base, runtime_key, gateway_id, environment_id = (os.environ[name] for name in NAMES)

try:
    with TokenTriage(base_url=base, api_key=runtime_key) as connection:
        client = connection.openai(gateway_id=gateway_id, environment_id=environment_id)
        raw = client.chat.completions.with_raw_response.create(
            model="gateway",
            messages=[{"role": "user", "content": "Say hello in one sentence."}],
            max_tokens=128,
        )
        answer = raw.parse().choices[0].message.content
except (TokenTriageError, APIError) as error:
    # A configuration error is raised locally and never reaches the gateway.
    details = error_details(error)
    sys.exit(f"Request failed: category={details.category} code={details.code} "
             f"status={details.status} request_id={details.request_id}")
except Exception as error:
    sys.exit(f"Request failed: {error!r}")

request_id = raw.headers.get("x-tt-request-id")
model = raw.headers.get("x-tt-model")
print(answer)
print("Request ID:", request_id)
print("Model:", model)
if model in (None, "gateway"):
    print("Warning: x-tt-model is missing or still 'gateway'; check the gateway's applied release.",
          file=sys.stderr)
if request_id is None:
    sys.exit("No x-tt-request-id was returned; the response did not come from a TokenTriage gateway.")

# A successful run prints, for example:
#   Hello! How can I help you today?
#   Request ID: req_...
#   Model: <provider model chosen by the applied release>

How to tell it worked. The answer prints first. Then x-tt-request-id, which is the ID to quote when you ask for help, and x-tt-model, which holds the provider model the release selected. An x-tt-model that names a provider model rather than the string gateway is the evidence that routing happened. x-tt-release, x-tt-gateway and x-tt-environment are on the same response if you want to record which configuration served the call.

This exact snippet runs in the SDK’s local HTTP checks. The packaged B1 call site in examples/acme-hr-portal has also completed ordinary and streamed requests against the private deployment, with the saved HR connection and priced customer receipts. That live application run is separate from execution of this shorter snippet.

3. Or from TypeScript

Run this in your backend, where the runtime key stays private.

import OpenAI from "openai";
import { TokenTriage, TokenTriageError, errorDetails, gatewayModel } from "tokentriage/client";

const names = ["TOKENTRIAGE_GATEWAY_BASE", "TOKENTRIAGE_RUNTIME_KEY",
  "TOKENTRIAGE_GATEWAY_ID", "TOKENTRIAGE_ENVIRONMENT_ID"];
const missing = names.filter((name) => !process.env[name]);
if (missing.length > 0) {
  console.error(`Set ${missing.join(", ")} before running this example.`);
  process.exit(1);
}
const [baseUrl, apiKey, gatewayId, environmentId] = names.map((name) => process.env[name] ?? "");

let connection: TokenTriage | undefined;
try {
  connection = new TokenTriage({ baseUrl, apiKey });
  const client = await connection.openai({ gatewayId, environmentId });
  const { data, response } = await client.chat.completions.create({
    model: gatewayModel,
    messages: [{ role: "user", content: "Say hello in one sentence." }],
    max_tokens: 128,
  }).withResponse();
  const requestId = response.headers.get("x-tt-request-id");
  const model = response.headers.get("x-tt-model");
  console.log(data.choices[0]?.message.content);
  console.log("Request ID:", requestId);
  console.log("Model:", model);
  if (model === null || model === "gateway") {
    console.warn("Warning: x-tt-model is missing or still 'gateway'; check the gateway's applied release.");
  }
  if (requestId === null) {
    console.error("No x-tt-request-id was returned; the response did not come from a TokenTriage gateway.");
    process.exitCode = 1;
  }
} catch (error) {
  if (error instanceof TokenTriageError || error instanceof OpenAI.APIError) {
    // A configuration error is raised locally and never reaches the gateway.
    const details = errorDetails(error);
    console.error(`Request failed: category=${details.category} code=${details.code} `
      + `status=${details.status} request_id=${details.requestId}`);
  } else {
    console.error("Request failed:", error);
  }
  process.exitCode = 1;
} finally {
  connection?.close();
}

// A successful run prints, for example:
//   Hello! How can I help you today?
//   Request ID: req_...
//   Model: <provider model chosen by the applied release>

gatewayModel is the exported sentinel, so you import the string rather than retyping it. The same three identities come back on response.headers. This exact snippet runs in the SDK’s local HTTP checks. The stock OpenAI integration in sdk/examples/existing_openai.ts has separately served live A1 requests, including after process replacement with the same retained runtime key.

If your client insists on an apiKey

The official OpenAI constructor wants a nonempty apiKey, and in saved mode that value is ours. The SDK supplies the runtime key itself, so pass no apiKey: one you pass is read as a provider key aimed at the header the runtime key owns, and the SDK declines to build the request with provider_key_not_allowed before anything reaches the network.

4. Read the refusal you got instead

Saved mode decides one credential arrangement and one body shape, so a mistake comes back as a named code rather than as a provider error. These are the seven you are most likely to meet, each with its fix.

Code	Status	The likely cause	What to change
`provider_credential_not_allowed`	401	a provider key travelled with the request, on `Authorization` or on a carrier such as `X-Api-Key`	remove it; the stored connection supplies the provider credential
`ambiguous_credential`	400	two credentials named at once: `Authorization` sent twice, or a runtime key beside `X-Tokentriage-Key`	send the runtime key once, on `Authorization`, and drop the legacy account header
`gateway_required`	400	no gateway selector on the request	set `X-Tokentriage-Gateway`, or pass `gatewayId` / `gateway_id`; there is no default gateway
`gateway_not_found`	404	the gateway is unknown, archived, or outside this credential’s allowlist	check the ID against the application’s gateway page, then the credential’s allowlist
`environment_not_allowed`	403	the gateway is allowed but the environment is not the credential’s own	send the credential’s environment, or mint a credential for the environment you meant
`model_override_unsupported`	422	the body named a provider model instead of the sentinel	send `"model": "gateway"` and let the release choose
`no_eligible_target`	422	no target in the gateway’s deployed route can serve this endpoint or these request parameters, for example an embeddings model on a chat route	publish and deploy a release whose route includes a target model that supports this endpoint, then retry

Two of those answers are less specific than they look, for a reason. gateway_not_found is the one answer for unknown, archived and another organization’s gateways alike, so no request can be used to discover which gateway IDs exist. ambiguous_credential is a 400 rather than a 401 because nothing about either credential was judged: the request itself is malformed.

A 503 naming release_unavailable is a different class again. It says the gateway holds no live lease on an applied release, so check deployment status before another attempt. It is not a credential problem. If a transport failure leaves a paid outcome unknown, retain its request ID and inspect the receipt before resending; do not silently replay it.

Structured error metadata reads the category, status and request ID off the exception the provider client raises.

What this page does not cover

Streaming, async clients and keeping your own transport are on provider clients and streaming. Request-supplied mode, where a caller sends a provider key on X-Tokentriage-Provider-Key, is gateway configuration rather than a client choice, and the hosted product does not admit it today. Keep the request ID to retrieve the scoped customer receipt; traffic and spend reads explain its usage, release and cost states.