# Provider clients and streaming

Use official provider clients with async requests and streams.

Source: https://tokentriage.com/docs/sdks/adapters

---
import NextSteps from '../../../components/NextSteps.astro';

[Install the local SDK update](/docs/sdks/install) and set your environment.
These configured APIs are unreleased.

The snippets below pass a provider key to the client, which is the pre-Foundation
managed shape. On a saved-mode gateway there is no provider key to pass: drop that
argument and give the binding instead, `connection.async_openai(gateway_id="gw_...",
environment_id="env_...")` in Python or `connection.openai({ gatewayId,
environmentId })` in TypeScript, with `"gateway"` as the model.
[Your first routed request](/docs/sdks/first-request) walks one call end to end.

## Async Python

{/* sdk-example:async_request.py */}
```python
import asyncio
import os
import sys

from tokentriage.client import TokenTriage

NAMES = ("TOKENTRIAGE_GATEWAY_BASE", "TOKENTRIAGE_RUNTIME_KEY",
         "TOKENTRIAGE_GATEWAY_ID", "TOKENTRIAGE_ENVIRONMENT_ID")
missing = [name for name in NAMES if not os.environ.get(name)]
if missing:
    sys.exit(f"Set {', '.join(missing)} before running this example.")
base, runtime_key, gateway_id, environment_id = (os.environ[name] for name in NAMES)


async def main() -> None:
    async with TokenTriage(base_url=base, api_key=runtime_key) as connection:
        try:
            client = connection.async_openai(gateway_id=gateway_id, environment_id=environment_id)
            response = await client.chat.completions.with_raw_response.create(
                model="gateway",
                messages=[{"role": "user", "content": "Say hello in one sentence."}],
            )
        except Exception as error:
            details = connection.error_details(error)
            sys.exit(f"Request failed: code={details.code} status={details.status} "
                     f"request_id={details.request_id}")
        print(response.parse().choices[0].message.content)
        print("Request ID:", connection.metadata(response).request_id)


asyncio.run(main())
```
{/* sdk-example:end */}

## Python streaming

Headers carry the request ID before the stream finishes.
The context manager closes it when the loop exits early.

{/* sdk-example:stream.py */}
```python
import asyncio
import os
import sys

from openai import APIError
from tokentriage.client import TokenTriage, TokenTriageError, error_details

NAMES = ("TOKENTRIAGE_GATEWAY_BASE", "TOKENTRIAGE_RUNTIME_KEY",
         "TOKENTRIAGE_GATEWAY_ID", "TOKENTRIAGE_ENVIRONMENT_ID")
missing = [name for name in NAMES if not os.environ.get(name)]
if missing:
    sys.exit(f"Set {', '.join(missing)} before running this example.")
base, runtime_key, gateway_id, environment_id = (os.environ[name] for name in NAMES)


async def main() -> None:
    try:
        async with TokenTriage(base_url=base, api_key=runtime_key) as connection:
            client = connection.async_openai(gateway_id=gateway_id, environment_id=environment_id)
            async with client.chat.completions.with_streaming_response.create(
                model="gateway",
                messages=[{"role": "user", "content": "Say hello in one sentence."}],
                stream=True,
            ) as response:
                print("Request ID:", connection.metadata(response).request_id)
                stream = await response.parse()
                async for chunk in stream:
                    if chunk.choices:
                        print(chunk.choices[0].delta.content or "", end="", flush=True)
            print()
    except (TokenTriageError, APIError) as error:
        # A configuration error is raised locally and never reaches the gateway.
        details = error_details(error)
        sys.exit(f"Request failed: category={details.category} code={details.code} "
                 f"status={details.status} request_id={details.request_id}")
    except Exception as error:
        sys.exit(f"Request failed: {error!r}")


asyncio.run(main())
```
{/* sdk-example:end */}

## TypeScript streaming

{/* sdk-example:stream.ts */}
```ts
import OpenAI from "openai";
import { TokenTriage, TokenTriageError, errorDetails, gatewayModel } from "tokentriage/client";

const names = ["TOKENTRIAGE_GATEWAY_BASE", "TOKENTRIAGE_RUNTIME_KEY",
  "TOKENTRIAGE_GATEWAY_ID", "TOKENTRIAGE_ENVIRONMENT_ID"];
const missing = names.filter((name) => !process.env[name]);
if (missing.length > 0) {
  console.error(`Set ${missing.join(", ")} before running this example.`);
  process.exit(1);
}
const [baseUrl, apiKey, gatewayId, environmentId] = names.map((name) => process.env[name] ?? "");

let connection: TokenTriage | undefined;
try {
  connection = new TokenTriage({ baseUrl, apiKey });
  const client = await connection.openai({ gatewayId, environmentId });
  const { data: stream, response } = await client.chat.completions.create({
    model: gatewayModel,
    messages: [{ role: "user", content: "Say hello in one sentence." }],
    stream: true,
  }).withResponse();
  console.log("Request ID:", connection.metadata(response).requestId);
  try {
    for await (const chunk of stream) {
      process.stdout.write(chunk.choices[0]?.delta.content ?? "");
    }
  } finally {
    stream.controller.abort();
  }
  console.log();
} catch (error) {
  if (error instanceof TokenTriageError || error instanceof OpenAI.APIError) {
    // A configuration error is raised locally and never reaches the gateway.
    const details = errorDetails(error);
    console.error(`Request failed: category=${details.category} code=${details.code} `
      + `status=${details.status} request_id=${details.requestId}`);
  } else {
    console.error("Request failed:", error);
  }
  process.exitCode = 1;
} finally {
  connection?.close();
}
```
{/* sdk-example:end */}

Provider chunks and tool-call fields keep their official types.
Do not replay a stream after delivering part of it to your application.

## Keep your constructor or transport

Python provides `openai_options(http_client=...)` and
`async_openai_options(http_client=...)`. TypeScript provides
`openaiOptions({ fetch })`. Pass the result to the official OpenAI constructor.

Both options APIs take the saved-mode binding beside your transport:
`openai_options(gateway_id=..., environment_id=..., http_client=...)` and
`openaiOptions({ gatewayId, environmentId, fetch })`.

The connection borrows your supplied transport. Your application owns its cleanup.

`wrap_openai()` and `wrapOpenAI()` support factory-created clients.
Arbitrary third-party instances may hide custom transports; unsupported wrapping
raises a configuration error. Construct them with the options API instead.

## Choose how to activate

| Mode | Routing | Observation |
|---|---|---|
| Configured factory/options with a saved-mode binding | the gateway's applied release picks the model | response metadata, including the release and model it chose |
| Configured factory/options | Assigned gateway | Response metadata and optional local observations |
| Legacy `init()` and headers | Your existing destination | Labels and receipts |
| Local router | In-process policy | Local observations and optional Central export |

Automatic bootstrap for arbitrary provider clients is unsupported.
Use the factory or options API for explicit activation.

<NextSteps items={[
  { title: 'First routed request', href: '/docs/sdks/first-request' },
  { title: 'Provider clients and streaming', href: '/docs/sdks/adapters' }
]} />