Product

Everything you need to see — and safely cut — LLM spend.

Six pillars, one binary, one source of truth. Every surface below is the real dashboard running in demo mode against a synthetic ledger — labeled as such, so you always know sample data from the real thing.

Cost analytics & optimization

Where every dollar goes — and what’s safe to cut

Every request is costed and attributed by model, token type (input / output / cache / reasoning), trace, agent, and MCP tool. The optimizer turns that ledger into recommendations — and grades each one honestly.

Three savings verdicts: PROVEN (measured on real post-change traffic — never a forward-looking recommendation), ESTIMATED (a modeled projection), or UNPROVABLE (a disclosed refusal).
Provider-bill reconciliation: match an invoice against the ledger; the residual shows as “unexplained,” never zeroed to force agreement.
FinOps-ready: hourly rollups, budget burn-down, chargeback and FOCUS exports.

Recommendations view: cache-opportunity cards with projected savings marked ESTIMATED, repeat-rate stats, method provenance (cache-shadow-would-hit-v1), and an evidence link. — Recommendations are graded — here, ESTIMATED cache savings with the method and evidence shown. A recommendation can never be PROVEN, and that is by design.

Quality-proven routing

Routing that proves it beat the baseline — or admits it can’t

A typed, deterministic policy language decides each route and explains why. The honesty verdict answers one question: does routing beat the zero-router baseline at its own cost?

Eleven routing strategies ship — complexity thresholds, cost-aware selection, A/B/canary, budget degrade, ordered fallback, judge-graded cascades, and learned/bandit routing.
Shadow evaluation computes the counterfactual without touching a response — so you see the savings before you enforce them.
It refuses to fabricate a win. With only served-model labels it can say routing sits ON the frontier; only judged shadow duplication can show a BEAT. Otherwise: an honest refusal, never a number.

Honesty center showing CANNOT_MEASURE and 'Frontier refused — no measurable delta: insufficient labels (2 of 8) to judge routing.' — The Honesty center refusing to claim a routing win: “Frontier refused — no measurable delta.” That refusal, never a fabricated number, is the whole value of the tool.

The trust dashboard & provenance

Every number traces to one decision log

Each proxied request writes one row to a JSONL decision log. Every surface derives from it, and two surfaces may never disagree. Amounts render through a typed Money component stamped with the pricing snapshot.

UNPRICED is not zero. An unknown model yields a null cost and an excluded-row count — never a misleading $0.00.
EST flags mark any estimated figure; pricing is stamped source@date#sha7.
“View as table” on every chart, so any pixel can be checked against the rows behind it.

Pricing view triaging UNPRICED models — showing which models have no price and how unpriced spend is excluded rather than shown as zero. — Pricing surface: UNPRICED models triaged honestly. “We spent nothing on this model” and “we don’t know what we spent” are opposite facts — only one is good news.

Multi-provider breadth

Four wire dialects in code. Every provider as data.

The binary compiles four wire drivers — OpenAI, Anthropic, Bedrock, and Gemini. Every OpenAI-compatible upstream rides the OpenAI driver with zero new code: adding one is N+1-by-config.

OpenAI
Anthropic
Azure OpenAI
AWS Bedrock
Google Gemini / Vertex
Groq
Together AI
Local / self-hosted (vLLM, Ollama)

Opt-in cross-dialect translation (OpenAI ↔ Anthropic, off by default) discloses every dropped parameter rather than silently swallowing it.

Providers view listing configured upstreams with their health and pricing status. — The Providers surface: every configured upstream, its dialect, and its pricing lineage in one place.

Governance, virtual keys & budgets

Limits that fail open — and say so

Mint virtual keys (hashed, shown once, stripped before upstream) and enforce per-tenant limits across an org / team / user / key hierarchy. Off by default; shadow-first when you turn it on.

Six limit types: budget (USD), token & request quotas, RPM, TPM, and concurrency.
Row equals wire: one function stamps every refusal (401/403/429/503) into both the HTTP response and the ledger row.
Fail-open with disclosure: a dead accountant suspends enforcement and discloses it — never a fake 429. fail: closed is an honest opt-in.

Governance tenant view showing an org/team/key hierarchy with per-node limits and budgets. — Tenant governance: template limits with per-node overrides across the hierarchy, all disclosed in the ledger.

Deployment & air-gap

One CGO-free binary. No sidecar. No data files.

The pricing table is compiled in via go:embed; the dashboard SPA is embedded too. Cross-compiles to Linux, Windows, and macOS with no C toolchain.

Binary: ≈10.7 MB static (linux/amd64, CGO-free)
Container: FROM scratch, ≈11 MB (estimated), non-root
Deploy: Canonical Helm chart, Kustomize, Compose, Terraform
Secrets: References only (env-var names) — never values
Network: Air-gap profiles dial nothing external by default

Response caching

Off by default, shadow-first, byte-faithful, and honestly costed. The default exact/normalized tier is fully air-gapped — pure Go, dials nothing. Designed to be an honestLLM-router cache: it never claims a hit it can’t prove.

Fail-open, always

A broken ledger must never break your agents. Any internal error fails open to a verbatim passthrough, and the proxied bytes equal going direct — streaming included.

Read the docs, or read the code

Everything here ships in one Apache-2.0 binary. Start with the quickstart, or browse the source.

Read the docs Browse the source