Your first live request

In this tutorial we’ll send one ordinary chat request through a hosted gateway and read back which release served it and what it cost. Your application uses a TokenTriage runtime key; the provider key stays in the workspace’s saved connection.

Start with your company and first application, an OpenAI account with API access, and the local Python SDK build. You need someone with permission to manage connections, edit and deploy the gateway, and issue runtime keys. Provider checks and the final request can spend money on your provider account.

1. Prepare the saved connection

Open Provider connections for the workspace. Follow Add a saved provider connection to store an OpenAI key, check it against an eligible model, activate its candidate version, and grant the connection to this application in Development for application traffic.

Continue when the connection shows an active version and the application/environment/purpose grant is present. An accepted check alone does not activate a version or grant its use. Record the check’s attributed cost; an unknown cost is not a free check.

2. Confirm an applied release

Open the application’s gateway and choose Development, then Configure. The first request needs one eligible OpenAI model choice and a default route. Bind the OpenAI slot in Provider connections to the active connection from step 1, then save the draft.

Use the gateway builder’s release and promotion steps to review the destination bindings and deploy. Continue only when the deployment status reports an applied release. A saved draft or a desired release still being applied is not evidence that requests can use it. Keep the gateway ID, environment ID and assigned gateway base URL.

3. Obtain a ready runtime key

Have the application’s key manager create a service identity and a runtime credential for this gateway in Development. The hosted credential commands return the secret once and report its readiness. Wait for readiness: ready before using it.

Store that ttv_ secret in your application’s secret store and load these environment variables in the backend process running the example:

Variable	Value
TOKENTRIAGE_GATEWAY_BASE	The assigned gateway base URL
TOKENTRIAGE_RUNTIME_KEY	The ready runtime credential
TOKENTRIAGE_GATEWAY_ID	This gateway’s gw_ ID
TOKENTRIAGE_ENVIRONMENT_ID	Development’s env_ ID

The runtime credential authorizes only its named gateways and environment. Keep it out of browser code. Do not put your provider key or a ttm_ management key in this variable.

4. Send one bounded request

Save this as first_request.py and run python first_request.py in that process. It asks for at most 128 output tokens. The packaged client supplies the runtime credential and selectors; model="gateway" lets the applied release select the provider model.

import os
import sys

from openai import APIError
from tokentriage.client import TokenTriage, TokenTriageError, error_details

NAMES = ("TOKENTRIAGE_GATEWAY_BASE", "TOKENTRIAGE_RUNTIME_KEY",
         "TOKENTRIAGE_GATEWAY_ID", "TOKENTRIAGE_ENVIRONMENT_ID")
missing = [name for name in NAMES if not os.environ.get(name)]
if missing:
    sys.exit(f"Set {', '.join(missing)} before running this example.")
base, runtime_key, gateway_id, environment_id = (os.environ[name] for name in NAMES)

try:
    with TokenTriage(base_url=base, api_key=runtime_key) as connection:
        client = connection.openai(gateway_id=gateway_id, environment_id=environment_id)
        raw = client.chat.completions.with_raw_response.create(
            model="gateway",
            messages=[{"role": "user", "content": "Say hello in one sentence."}],
            max_tokens=128,
        )
        answer = raw.parse().choices[0].message.content
except (TokenTriageError, APIError) as error:
    # A configuration error is raised locally and never reaches the gateway.
    details = error_details(error)
    sys.exit(f"Request failed: category={details.category} code={details.code} "
             f"status={details.status} request_id={details.request_id}")
except Exception as error:
    sys.exit(f"Request failed: {error!r}")

request_id = raw.headers.get("x-tt-request-id")
model = raw.headers.get("x-tt-model")
print(answer)
print("Request ID:", request_id)
print("Model:", model)
if model in (None, "gateway"):
    print("Warning: x-tt-model is missing or still 'gateway'; check the gateway's applied release.",
          file=sys.stderr)
if request_id is None:
    sys.exit("No x-tt-request-id was returned; the response did not come from a TokenTriage gateway.")

# A successful run prints, for example:
#   Hello! How can I help you today?
#   Request ID: req_...
#   Model: <provider model chosen by the applied release>

The answer prints first, followed by Request ID and Model. Keep the request ID. The model should be the selected provider model, rather than the gateway sentinel. The TypeScript version uses the same contract.

5. Read the receipt and cost

Use a management credential with metadata and cost-read access for this separate read. The runtime key cannot read management records. With TOKENTRIAGE_MANAGEMENT_KEY loaded securely, run the following command after replacing the three uppercase values:

tokentriage cloud request get --base-url https://app.tokentriage.com --org ORG_ID --workspace WORKSPACE_ID --request REQUEST_ID --json

The returned record identifies the gateway, environment, applied release, provider attempt, usage and cost state. Compare its release with the applied status from step 2. A priced cost is an integer in nano-USD: divide by one billion for dollars. Keep unpriced and unavailable outcomes distinct from zero. Read your gateway’s traffic and spend explains those states and the corresponding SDK reads.

The exact Python and TypeScript tutorial sources are executed against a local HTTP fixture in the SDK checks. Live acceptance separately exercised the packaged TypeScript and Python applications with real saved connections, scoped runtime keys and priced receipts. The fixture’s synthetic response is not a provider bill.

If the request fails

Keep the request ID and named error before changing anything. These recoveries preserve the connection and release you configured:

Result	Recovery
Credential issuance outcome is uncertain	Read the operation’s status. Do not repeat mint blindly: it can create a second credential. A lost one-time secret needs revocation and replacement.
gateway_required or environment_not_allowed	Check the two selector variables against the credential’s allowlist and environment.
model_override_unsupported	Keep model="gateway"; choose the provider model in the gateway configuration.
no_eligible_target	Publish and deploy a release whose route includes a target model that supports this endpoint, then retry.
release_unavailable	Check the deployment status and applied release before another request. A saved draft is insufficient.
request_outside_qualification	Reduce the request to the supported text request and explicit output cap; consult the named model and format limits before resending.
Connection is unavailable or unauthorized	Ask the connection manager to check the active version and the exact application/environment/purpose grant.
A timeout or transport failure leaves the outcome unknown	Read the request receipt if an ID was returned. Preserve possible spend; do not automatically replay an uncertain paid call.

This tutorial sends one ordinary request. It does not test streaming, fallback, every guardrail or a production promotion. A successful answer proves this request’s path; it does not grant additional gateways or environments.