How TokenTriage compares

A large language model (LLM) gateway sits between your application and model providers. When you compare gateways, start with the operating model: who runs it, who stores your provider API credentials, and whose provider accounts pay the bill.

OpenRouter is a hosted marketplace. Portkey and Helicone are hosted platforms with self-hosted gateway options. LiteLLM, Bifrost, and TokenTriage are primarily software you run in your own environment. That difference usually narrows the field faster than a feature checklist.

If you are in a hurry, start with your hardest constraint below. The product sections explain the trade-offs, and the migration links show what changing gateways involves.

How to read this comparison

We build TokenTriage, so this page reflects our perspective. Product descriptions were checked against vendor documentation in July and August 2026, with source links in each section. This market changes quickly, so check those sources before making a final decision.

The performance results are ours. They name the hardware, competitor versions, and limits of the test. Cost accuracy was a tie among the three gateways tested, and response quality was not tested.

Choose by constraint

Start with the requirement you cannot compromise on. Several of these answers are not TokenTriage.

if you care most about…	start with	why
using models without opening an account with every provider	OpenRouter	one marketplace account, one key, and one bill
a hosted management service with a UI	Portkey	gateway, guardrails, observability, and prompt tools managed in one platform
getting requests logged and visible quickly	Helicone	managed observability and an async logging option
the widest provider catalog	LiteLLM	100+ providers and the largest open-source community in this comparison
a Model Context Protocol (MCP) gateway or gateway plugins	Bifrost or Portkey	both provide gateway extensibility that TokenTriage does not
keeping credentials in your infrastructure without adding a database server	TokenTriage	one executable, environment-variable references, and local storage
checking the cost of one request against the rates used	TokenTriage	each costed row records its pricing source
previewing routing decisions before they change traffic	TokenTriage	shadow mode records the decision it would make without enforcing it
a guaranteed savings percentage before rollout	none of them	savings depend on your traffic and model mix

These are starting points, not overall rankings. Deployment, model access, and day-to-day operations still matter.

Three ways to run a gateway

Hosted means the vendor runs the service. Self-hosted means your team deploys and operates it. A marketplace also sells access to models, so you do not need a separate account with every provider. Some products offer more than one of these options.

product	usual starting point	where provider credentials live	what you operate
TokenTriage	self-hosted gateway	your environment; config stores environment-variable names, not secret values	one executable and local data files
LiteLLM	self-hosted proxy or Python library	your environment; config can contain a secret or reference an environment variable	the proxy, plus Postgres for keys and budgets
Portkey	hosted platform, with an open-source gateway option	hosted credentials are stored by Portkey; self-hosted credentials stay in your environment	nothing when hosted; a container when self-hosted
OpenRouter	hosted model marketplace	you use one OpenRouter key; OpenRouter owns the provider accounts	nothing
Helicone	hosted observability and gateway, with a self-hosted gateway option	the hosted proxy receives credentials; async logging leaves the provider call with you	nothing when hosted; a container when self-hosted
Bifrost	self-hosted gateway	your environment	one executable or container

For TokenTriage, “one executable” means the proxy, dashboard, price table, and local request history ship together. The Quickstart shows the command and its output.

Product-by-product trade-offs

TokenTriage

Best fit. TokenTriage suits teams that want to keep credentials and traffic in their own environment, see an auditable cost for each request, and introduce routing in stages. Its default mode records traffic in a local decision ledger without changing which model serves the request.

Trade-offs. The hosted control plane is live in private preview; public self-service signup is not open. You can contact us about access. Its provider catalog and community are smaller than LiteLLM’s, and it does not provide library mode, an MCP gateway, gateway plugins, or the same depth of load balancing and failover. It is beta software.

LiteLLM

Best fit. LiteLLM is the shortest path when your code calls many model providers and you want one client interface. It supports 100+ providers as a Python library or a deployed proxy, with virtual keys, budgets, spend tracking, guardrails, and an admin UI. See the provider catalog and proxy deployment guide.

How TokenTriage differs. LiteLLM uses Postgres for its key and budget features. Its config can store a provider secret inline or reference an environment variable. TokenTriage is proxy-only, uses an embedded SQLite file for governance, and accepts only the environment-variable reference.

Choose LiteLLM when catalog breadth, library mode, community size, ordered fallback chains, or more advanced load balancing matters more than operating one executable. The key-by-key mapping is in Migrate from LiteLLM.

Portkey

Best fit. Portkey suits a team that wants one vendor to run the management surface. Its hosted platform combines a gateway, routing, guardrails, observability, prompt tools, and enterprise controls. Saved configurations can be selected per request, so a team can change behavior in the UI without shipping application code. See the Portkey documentation.

How TokenTriage differs. TokenTriage offers a self-hosted gateway and a separate hosted control plane in private preview. For the self-hosted gateway, configuration lives in files you operate. The products also use “virtual key” differently: a Portkey virtual key references a provider credential held by Portkey, while a TokenTriage virtual key is an access token with budgets and permissions but no provider credential.

Choose Portkey when a hosted UI, prompt studio, managed retention, or per-request overrides are requirements. The header and configuration differences are in Migrate from Portkey.

OpenRouter

Best fit. OpenRouter is a marketplace: one account, one key, and one bill can reach hundreds of models across many providers. It can also route among providers that host the same model. See the quickstart and model-routing documentation.

How TokenTriage differs. TokenTriage does not sell model access or combine provider bills. You bring your own provider accounts and keys, then route among the models and providers you configured. That keeps purchasing and credentials under your control, but it requires an account with each provider you use.

Choose OpenRouter when immediate access to models, consolidated billing, or routing across several hosts of the same model matters most. The control mapping is in Migrate from OpenRouter.

Helicone

Best fit. Helicone starts with observability: get a request logged, visible, and debuggable, then add gateway features. It supports a proxy in the request path and async logging that leaves the provider call untouched. See the Helicone quickstart and gateway overview.

How TokenTriage differs. TokenTriage works only as a proxy in the request path. It writes the decision ledger as it serves each request, keeps that history locally, and builds its dashboard from the same local data. There is no async logging option.

Choose Helicone when a managed dashboard, hosted retention, or logging without putting a new proxy in the request path is the priority. There is no Helicone-specific migration guide yet; the base-URL change is covered in Migrate from OpenAI or Anthropic SDKs.

Bifrost

Best fit. Bifrost is a self-hosted gateway written in Go, distributed as an executable or container. It focuses on low overhead and includes an MCP gateway and plugin system. See the Bifrost documentation.

How TokenTriage differs. Both products have a similar deployment shape. Bifrost emphasizes throughput and gateway extensibility. TokenTriage emphasizes the cost and routing record for each request: which price was used, which candidates were considered, and why one was selected.

Choose Bifrost when brokering MCP servers or extending the gateway through plugins is a core requirement. TokenTriage can record MCP tool use, but it does not broker MCP servers. There is no Bifrost-specific migration guide yet; OpenAI-compatible traffic starts with a base-URL change.

Where TokenTriage is behind today

These are current product gaps, not roadmap commitments.

gap	who is stronger	what that means
provider catalog and library support	LiteLLM	TokenTriage has 95 built-in provider descriptors, a smaller catalog, and no library mode
load balancing and failover	LiteLLM	no per-model ordered fallback chain, least-busy selection, active health probes, or request hedging
per-request overrides	Portkey	no request-level provider selector, cache namespace, timeout, or custom-host control
marketplace access and provider filters	OpenRouter	no consolidated billing or marketplace-wide routing across third-party providers
managed operations	Portkey and Helicone	hosted TokenTriage access is private preview; public self-service signup is not open
MCP and plugin ecosystem	Bifrost and Portkey	TokenTriage records tool use but does not broker MCP servers or load gateway plugins
ecosystem and maturity	LiteLLM and the hosted platforms	a smaller community and a beta product

What the 95-provider count does and does not mean

TokenTriage publishes support per provider and endpoint. The support matrix names the automated test behind each gated cell and marks the rest as declared (no e2e gate). Seven provider descriptors have no gated cell. Verification uses mocks rather than provider live endpoints, so the count describes built-in integrations, not a claim that every endpoint was tested against every provider.

Streaming guardrails and agent attribution also have limits, but the products on this page were not compared on those axes. See Features at a glance for those product-level limitations.

What moving involves

Most gateway migrations begin with a base-URL change. The operating model determines what else has to move.

from	the additional work
LiteLLM	map proxy config, keys, budgets, routing, and fallback behavior; use Migrate from LiteLLM
Portkey	replace saved hosted configurations and Portkey request headers; use Migrate from Portkey
OpenRouter	add your own provider accounts and replace marketplace-specific routing controls; use Migrate from OpenRouter
Helicone	change the proxy base URL; async logging has no TokenTriage equivalent
Bifrost	change the base URL for OpenAI-compatible traffic; no dedicated guide exists yet

Performance evidence

Performance matters after the operating model fits. In one run on an AWS z1d.metal bare-metal instance, each gateway received four cores and called the same zero-delay mock provider. The pinned competitors were LiteLLM v1.94.0 and Bifrost v1.6.6.

TokenTriage added 0.148 ms of gateway overhead, with a 95% confidence interval of [0.146, 0.150]. Bifrost added 0.392 ms, and LiteLLM added 9.245 ms; the intervals did not overlap. TokenTriage and Bifrost each cleared the harness ceiling of 4,000 requests per second. LiteLLM plateaued near 670. This test isolates gateway overhead, not the model-provider latency your users experience.

What the benchmark does not tell you

• End-to-end latency. The mock provider answered immediately. A model provider usually takes hundreds of milliseconds, so these numbers do not predict how fast the application feels.
• Maximum Bifrost or TokenTriage throughput. Both reached the 4,000-request harness ceiling without saturating, so their limits were not measured.
• Savings or response quality. Both depend on your traffic and model choices. Measure routing in shadow mode before enforcing it.
• Hosted Portkey or Helicone cost accuracy. Their self-hosted gateways do not expose a local per-request cost in the form the harness scores, so neither product was graded on that axis.
• Independent reproduction. A clean-clone run by someone unaffiliated with TokenTriage is still pending.

The historical same-run methodology and reproduction command remain in the repository. Windows loopback measurements, binary size, and continuous-integration budgets are documented in Benchmarks.