# How TokenTriage compares
Choose among TokenTriage, LiteLLM, Portkey, OpenRouter, Helicone, and Bifrost based on hosting, credentials, operations, and model access.
Source: https://tokentriage.com/docs/introduction/how-tokentriage-compares
---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
A large language model (LLM) gateway sits between your application and model providers. When you
compare gateways, start with the operating model: who runs it, who stores your provider API
credentials, and whose provider accounts pay the bill.
OpenRouter is a hosted marketplace. Portkey and Helicone are hosted platforms with self-hosted
gateway options. LiteLLM, Bifrost, and TokenTriage are primarily software you run in your own
environment. That difference usually narrows the field faster than a feature checklist.
If you are in a hurry, start with your hardest constraint below. The product sections explain the
trade-offs, and the migration links show what changing gateways involves.
## How to read this comparison
We build TokenTriage, so this page reflects our perspective. Product descriptions were checked
against vendor documentation in July and August 2026, with source links in each section. This
market changes quickly, so check those sources before making a final decision.
The performance results are ours. They name the hardware, competitor versions, and limits of the
test. Cost accuracy was a tie among the three gateways tested, and response quality was not tested.
## Choose by constraint
Start with the requirement you cannot compromise on. Several of these answers are not
TokenTriage.
| if you care most about... | start with | why |
|---|---|---|
| using models without opening an account with every provider | **OpenRouter** | one marketplace account, one key, and one bill |
| a hosted management service with a UI | **Portkey** | gateway, guardrails, observability, and prompt tools managed in one platform |
| getting requests logged and visible quickly | **Helicone** | managed observability and an async logging option |
| the widest provider catalog | **LiteLLM** | 100+ providers and the largest open-source community in this comparison |
| a Model Context Protocol (MCP) gateway or gateway plugins | **Bifrost** or **Portkey** | both provide gateway extensibility that TokenTriage does not |
| keeping credentials in your infrastructure without adding a database server | **TokenTriage** | one executable, environment-variable references, and local storage |
| checking the cost of one request against the rates used | **TokenTriage** | each costed row records its pricing source |
| previewing routing decisions before they change traffic | **TokenTriage** | shadow mode records the decision it would make without enforcing it |
| a guaranteed savings percentage before rollout | **none of them** | savings depend on your traffic and model mix |
These are starting points, not overall rankings. Deployment, model access, and day-to-day
operations still matter.
## Three ways to run a gateway
**Hosted** means the vendor runs the service. **Self-hosted** means your team deploys and operates
it. A **marketplace** also sells access to models, so you do not need a separate account with every
provider. Some products offer more than one of these options.
| product | usual starting point | where provider credentials live | what you operate |
|---|---|---|---|
| **TokenTriage** | self-hosted gateway | your environment; config stores environment-variable names, not secret values | one executable and local data files |
| **LiteLLM** | self-hosted proxy or Python library | your environment; config can contain a secret or reference an environment variable | the proxy, plus Postgres for keys and budgets |
| **Portkey** | hosted platform, with an open-source gateway option | hosted credentials are stored by Portkey; self-hosted credentials stay in your environment | nothing when hosted; a container when self-hosted |
| **OpenRouter** | hosted model marketplace | you use one OpenRouter key; OpenRouter owns the provider accounts | nothing |
| **Helicone** | hosted observability and gateway, with a self-hosted gateway option | the hosted proxy receives credentials; async logging leaves the provider call with you | nothing when hosted; a container when self-hosted |
| **Bifrost** | self-hosted gateway | your environment | one executable or container |
For TokenTriage, “one executable” means the proxy, dashboard, price table, and local request history
ship together. The [Quickstart](/docs/get-started/quickstart) shows the command and its output.
## Product-by-product trade-offs
### TokenTriage
**Best fit.** TokenTriage suits teams that want to keep credentials and traffic in their own
environment, see an auditable cost for each request, and introduce routing in stages. Its default
mode records traffic in a local decision ledger without changing which model serves the request.
**Trade-offs.** The hosted control plane is live in private preview; public self-service signup
is not open. You can [contact us](/contact/) about access. Its provider catalog and community are
smaller than LiteLLM's, and it does not provide library mode, an MCP gateway, gateway plugins, or
the same depth of load balancing and failover. It is beta software.
### LiteLLM
**Best fit.** LiteLLM is the shortest path when your code calls many model providers and you want
one client interface. It supports 100+ providers as a Python library or a deployed proxy, with
virtual keys, budgets, spend tracking, guardrails, and an admin UI. See the
[provider catalog](https://docs.litellm.ai/docs/providers) and
[proxy deployment guide](https://docs.litellm.ai/docs/proxy/deploy).
**How TokenTriage differs.** LiteLLM uses Postgres for its key and budget features. Its config can
store a provider secret inline or reference an environment variable. TokenTriage is proxy-only,
uses an embedded SQLite file for governance, and accepts only the environment-variable reference.
**Choose LiteLLM when** catalog breadth, library mode, community size, ordered fallback chains, or
more advanced load balancing matters more than operating one executable. The key-by-key mapping is
in [Migrate from LiteLLM](/docs/migrate/litellm).
### Portkey
**Best fit.** Portkey suits a team that wants one vendor to run the management surface. Its hosted
platform combines a gateway, routing, guardrails, observability, prompt tools, and enterprise
controls. Saved configurations can be selected per request, so a team can change behavior in the
UI without shipping application code. See the [Portkey documentation](https://portkey.ai/docs).
**How TokenTriage differs.** TokenTriage offers a self-hosted gateway and a separate hosted
control plane in private preview. For the self-hosted gateway, configuration lives in files you
operate. The products also use “virtual key” differently: a Portkey virtual key references a
provider credential held by Portkey, while a TokenTriage virtual key is an access token with
budgets and permissions but no provider credential.
**Choose Portkey when** a hosted UI, prompt studio, managed retention, or per-request overrides are
requirements. The header and configuration differences are in
[Migrate from Portkey](/docs/migrate/portkey).
### OpenRouter
**Best fit.** OpenRouter is a marketplace: one account, one key, and one bill can reach hundreds of
models across many providers. It can also route among providers that host the same model. See the
[quickstart](https://openrouter.ai/docs/quickstart) and
[model-routing documentation](https://openrouter.ai/docs/features/model-routing).
**How TokenTriage differs.** TokenTriage does not sell model access or combine provider bills. You
bring your own provider accounts and keys, then route among the models and providers you configured.
That keeps purchasing and credentials under your control, but it requires an account with each
provider you use.
**Choose OpenRouter when** immediate access to models, consolidated billing, or routing across
several hosts of the same model matters most. The control mapping is in
[Migrate from OpenRouter](/docs/migrate/openrouter).
### Helicone
**Best fit.** Helicone starts with observability: get a request logged, visible, and debuggable,
then add gateway features. It supports a proxy in the request path and async logging that leaves
the provider call untouched. See the [Helicone quickstart](https://docs.helicone.ai/getting-started/quick-start)
and [gateway overview](https://docs.helicone.ai/gateway/overview).
**How TokenTriage differs.** TokenTriage works only as a proxy in the request path. It writes the
decision ledger as it serves each request, keeps that history locally, and builds its dashboard
from the same local data. There is no async logging option.
**Choose Helicone when** a managed dashboard, hosted retention, or logging without putting a new
proxy in the request path is the priority. There is no Helicone-specific migration guide yet; the
base-URL change is covered in
[Migrate from OpenAI or Anthropic SDKs](/docs/migrate/openai-anthropic).
### Bifrost
**Best fit.** Bifrost is a self-hosted gateway written in Go, distributed as an executable or
container. It focuses on low overhead and includes an MCP gateway and plugin system. See the
[Bifrost documentation](https://docs.getbifrost.ai/).
**How TokenTriage differs.** Both products have a similar deployment shape. Bifrost emphasizes
throughput and gateway extensibility. TokenTriage emphasizes the cost and routing record for each
request: which price was used, which candidates were considered, and why one was selected.
**Choose Bifrost when** brokering MCP servers or extending the gateway through plugins is a core
requirement. TokenTriage can record MCP tool use, but it does not broker MCP servers. There is no
Bifrost-specific migration guide yet; OpenAI-compatible traffic starts with a base-URL change.
## Where TokenTriage is behind today
These are current product gaps, not roadmap commitments.
| gap | who is stronger | what that means |
|---|---|---|
| provider catalog and library support | LiteLLM | TokenTriage has 95 built-in provider descriptors, a smaller catalog, and no library mode |
| load balancing and failover | LiteLLM | no per-model ordered fallback chain, least-busy selection, active health probes, or request hedging |
| per-request overrides | Portkey | no request-level provider selector, cache namespace, timeout, or custom-host control |
| marketplace access and provider filters | OpenRouter | no consolidated billing or marketplace-wide routing across third-party providers |
| managed operations | Portkey and Helicone | hosted TokenTriage access is private preview; public self-service signup is not open |
| MCP and plugin ecosystem | Bifrost and Portkey | TokenTriage records tool use but does not broker MCP servers or load gateway plugins |
| ecosystem and maturity | LiteLLM and the hosted platforms | a smaller community and a beta product |
TokenTriage publishes support per provider and endpoint. The support matrix names the automated
test behind each gated cell and marks the rest as `declared (no e2e gate)`. Seven provider
descriptors have no gated cell. Verification uses mocks rather than provider live endpoints, so the
count describes built-in integrations, not a claim that every endpoint was tested against every
provider.
Streaming guardrails and agent attribution also have limits, but the products on this page were not
compared on those axes. See [Features at a glance](/docs/introduction/features#known-limitations)
for those product-level limitations.
## What moving involves
Most gateway migrations begin with a base-URL change. The operating model determines what else has
to move.
| from | the additional work |
|---|---|
| **LiteLLM** | map proxy config, keys, budgets, routing, and fallback behavior; use [Migrate from LiteLLM](/docs/migrate/litellm) |
| **Portkey** | replace saved hosted configurations and Portkey request headers; use [Migrate from Portkey](/docs/migrate/portkey) |
| **OpenRouter** | add your own provider accounts and replace marketplace-specific routing controls; use [Migrate from OpenRouter](/docs/migrate/openrouter) |
| **Helicone** | change the proxy base URL; async logging has no TokenTriage equivalent |
| **Bifrost** | change the base URL for OpenAI-compatible traffic; no dedicated guide exists yet |
## Performance evidence
Performance matters after the operating model fits. In one run on an AWS `z1d.metal` bare-metal
instance, each gateway received four cores and called the same zero-delay mock provider. The pinned
competitors were LiteLLM v1.94.0 and Bifrost v1.6.6.
TokenTriage added **0.148 ms** of gateway overhead, with a 95% confidence interval of
`[0.146, 0.150]`. Bifrost added **0.392 ms**, and LiteLLM added **9.245 ms**; the intervals did not
overlap. TokenTriage and Bifrost each cleared the harness ceiling of 4,000 requests per second.
LiteLLM plateaued near 670. This test isolates gateway overhead, not the model-provider latency your
users experience.
TokenTriage, LiteLLM, and Bifrost produced the same request costs against provider-documented rates.
That axis was a tie. Response quality and savings were not measured.
- **End-to-end latency.** The mock provider answered immediately. A model provider usually takes
hundreds of milliseconds, so these numbers do not predict how fast the application feels.
- **Maximum Bifrost or TokenTriage throughput.** Both reached the 4,000-request harness ceiling
without saturating, so their limits were not measured.
- **Savings or response quality.** Both depend on your traffic and model choices. Measure routing
in shadow mode before enforcing it.
- **Hosted Portkey or Helicone cost accuracy.** Their self-hosted gateways do not expose a local
per-request cost in the form the harness scores, so neither product was graded on that axis.
- **Independent reproduction.** A clean-clone run by someone unaffiliated with TokenTriage is still
pending.
The historical same-run methodology and reproduction command remain in the repository.
Windows loopback measurements, binary size, and continuous-integration budgets are documented in
[Benchmarks](/docs/reference/benchmarks).