TokenTriage documentation Start locally, connect your app, and learn how to track costs, test routing, and run TokenTriage in production.
TokenTriage is a self-hosted service that sits between your app and its model providers. It records the cost of each request and lets you test lower-cost routing before you enable it.
Choose a starting point What do you want to do? Connect my app Change the base URL, send a live request, and confirm it passed through TokenTriage. Track and attribute costs See which team, feature, or customer generated each request and its cost. Route requests safely Test lower-cost models against your requirements before you send them live traffic. Set spending limits Set limits by tenant, feature, or key and understand why a request was blocked. Add guardrails Inspect prompts and responses for personal data, secrets, and unsafe content. Deploy to production Choose a topology, secure the service, and plan backups and upgrades. Browse all documentation Need a specific topic? Open the complete map, grouped by the kind of work you are doing.
Show all sections
Learn Understand the product, run it locally, and learn the terms used throughout the docs.
Build Connect your app and configure providers, routing, costs, limits, caching, and guardrails.
Routing Choose a model for each request, test the policy, and inspect every decision. Governance Set spending and rate limits by tenant, feature, or key. Providers & dialects Connect your current providers and handle provider-specific authentication, formats, and pricing. Cost & attribution Attribute each request's cost to a team, feature, or customer and assess its reliability. Guardrails Inspect prompts and responses for personal data, secrets, and unsafe content. Caching Reuse safe responses, measure the savings, and understand why a request missed the cache. Observability & tracing Trace one request, export telemetry, and understand the metrics TokenTriage records. Dashboard Turn on the dashboard, sign in, and learn what each screen and number means. Recipes Copy a focused configuration, complete one task, and verify the result. SDKs Instrument Python or TypeScript apps and trace costs back to the code that made each call.
Validate Measure model quality and decide when an automated score is reliable enough to use.
Run in production Deploy, secure, scale, back up, upgrade, and troubleshoot TokenTriage.
Look up & migrate Find exact commands and fields, or move from another gateway or SDK format.
API reference API reference Browse every Management API endpoint, schema, and error, with copyable request examples.