# Recommendations
What the detection loop proposes, why most proposals cannot be applied by a config change, and how to read a card before acting on it.
Source: https://tokentriage.com/docs/build/cost-optimization
---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';
import Screenshot from '../../../components/Screenshot.astro';
Cost analytics tells you where the money went. A recommendation goes further and says what to
do about it, which is a much harder claim: it is a statement about traffic that has not
happened yet.
The engine scans your ledger on an interval, looks for eight specific shapes of waste, and
produces a card for each opportunity it finds. What is worth understanding before you trust one
is not the list of shapes, which is
[the eight recommendation types](/docs/build/cost-recommendation-types). It is what a card is
allowed to claim, and the two quite different ways it can decline to claim anything.
## Nothing applies itself
There is no auto-apply switch, and there is no configuration leaf that would add one. The engine
detects, proves, and declines; a person applies.
That is a design position rather than a missing feature. A loop that could rewrite your routing
config on its own reading of a statistical bound would be making a quality decision on your
behalf, at a moment when nobody is looking at the evidence.
## A card, read end to end
Here is one from a live scan, trimmed to the parts you would read first.
```json
{
"id": "rec-routing-downgrade-pin",
"type": "routing-downgrade",
"scope": "decision.rule = \"pin\"",
"opportunity": {
"rule": "pin",
"slice_rows": 96,
"graded_rows": 11,
"passed_rows": 10,
"top_tier_usd": 61.212305,
"total_usd": 61.85027725,
"top_tier_share": 0.9896852160028111,
"tiers_seen": ["cheap", "mid"],
"candidate_tier_reason": "the 11 graded rows on this slice ran their shadow duplicate at 2 DIFFERENT tiers (cheap on 8, mid on 3), so the 90.9% pass rate is a blend and licenses no single tier; scope the recommendation to one tier before it can propose a routing change"
},
"projected": {
"usd": null,
"verdict": "UNPROVABLE",
"basis": {
"method": "shadow-measured-delta-v1",
"rows": 96,
"reason": "quality is not established on this slice: the Wilson lower bound over 11 graded rows (10 passing) is 0.623, below the 0.90 licensing bound. About 35 all-pass graded rows on this scope would license it; grade more shadow rows on this slice to collect them"
},
"baseline": { "kind": "measured-current", "desc": "vs this slice's actual recorded spend over the window" }
},
"proof": {
"quality_safety": "unknown",
"method": "shadow-grade-wilson-v1",
"labeled_rows": 11,
"pass_rate": 0.6226353745137962,
"bound": 0.9,
"required_rows": 35,
"guards": [
{ "name": "cache-trajectory", "passed": true, "params": { "cache_read_share": "0.0015", "threshold": "0.15" } },
{ "name": "retry-inflation", "passed": true, "params": { "candidate_failure_rate": "0.0909", "incumbent_failure_rate": "0.1818", "margin": "0.05" } }
]
},
"status": "open"
}
```
That capture is from a local sandbox against mock upstreams, so the dollars are arithmetic over
the mock's token counts. Four things are worth pulling out of it.
**The scope is re-derivable.** `decision.rule = "pin"` is filter text you can paste into a query
and get the same 96 rows back. A recommendation's population is checkable rather than merely asserted.
**The opportunity is separate from the claim.** `top_tier_share: 0.99` says almost all of this
rule's spend went to the most expensive tier. That is a fact about your ledger and it is true
regardless of what the card recommends.
**The projection declined.** Ten of eleven graded rows passed, which reads as a 91% pass rate.
Eleven rows is too few for that to mean anything, so the lower bound of the interval is 0.623,
under the 0.90 bar, and no dollar is rendered. The reason names the fix: about thirty-five
all-pass graded rows on this scope.
**Both guards ran and passed**, carrying the numbers they decided on. The cheaper tier's failure
rate came out *lower* than the incumbent's, so retry inflation was not a risk here.
`pass_rate` on the wire carries the **Wilson lower bound**, not the raw pass rate: `0.623`
here, where ten of eleven rows passed. It is a misleading field name. Read
`labeled_rows`, `passed_rows` and the reason text, and treat `pass_rate` as the bound it is.
## Two verdicts, and they do not blend
Every card carries a **quality** claim and an **impact** claim, judged separately and by
different machinery.
**Quality** answers *did the cheaper option answer as well?* It comes from graded shadow rows
with a statistical lower bound on the pass rate, or from a mechanism argument where the served
bytes are constructionally the incumbent's.
**Impact** answers *how many dollars?* That is the savings verdict.
They stay apart because they fail independently. A slice can have thoroughly established
quality and an `ESTIMATED` impact, which is the normal case rather than a contradiction —
knowing the cheap model is good enough says nothing about how much traffic will use it next
month. The chip on a card shows the **impact** verdict.
A recommendation is `ESTIMATED` at best. It projects measured per-row deltas over volume that
has not happened, and that is a model however strong the quality evidence. Only
[the post-apply measurement](/docs/build/cost-realized) produces a `PROVEN` dollar.
## Guards block; they do not discount
Some savings are sound arithmetic and still not worth taking, because the dollar does not show
what buying it costs. Three guards check for that, and a failing guard withholds the figure
rather than shrinking it.
An output cap is the clearest case. Capping completions at 512 tokens saves money, and if 15% of
a slice's answers would come back truncated, the saving is genuine and the product is different.
So the card is blocked with the arithmetic quoted (the truncated share, the row counts behind
it, and the 10% limit it exceeded) instead of rendering a smaller number that hides the trade.
A guard with nothing to compare on a slice is still evaluated and still passed, with its
emptiness recorded in its parameters. A skipped guard and a passing guard would be
indistinguishable on every surface, so skipping is not an option the engine has.
## Most cards cannot be applied, and that is the answer
Six of the eight types are **advisory**: they carry a recipe rather than a configuration
candidate, because the change is not something this config can express. Enrolling in a
provider's batch API, restructuring a prompt so its stable prefix caches, or capping output at
the call site all happen outside TokenTriage.
Two types are **actionable**, `routing-downgrade` and `cache-opportunity`, and carry a config
candidate. Applying one compiles it, validates it, diffs it, writes it through the same path a
manual edit uses, and audits it, returning the diff, the new generation and an audit id.
Ask to apply an advisory card and you get a `422` with the recipe. That refusal is the correct
answer rather than an error: the work exists, and it is yours to do somewhere else.
An actionable card can also refuse, for reasons of its own rather than of its type:
```console
$ curl -s -X POST http://127.0.0.1:9090/tt/api/v1/recommendations/rec-routing-downgrade-pin/apply
{"type":"/tt/api/problems/validation_failed","title":"Validation failed","status":422,"detail":"this recommendation cannot be compiled into a routing change because the 11 graded rows on this slice ran their shadow duplicate at 2 DIFFERENT tiers (cheap on 8, mid on 3), so the 90.9% pass rate is a blend and licenses no single tier; scope the recommendation to one tier before it can propose a routing change (this artifact type CAN compile a candidate, so this is a property of this artifact rather than of its type) No recipe was recorded on the artifact.","code":"validation_failed","request_id":"req_ebedacbd789fd5aa"}
```
Note the parenthesis: the response distinguishes "this type cannot compile a candidate" from
"this particular card cannot", which are different problems with different fixes.
## Finding nothing is an answer
The most common state of this surface is empty, and an empty list is where a detection loop
usually starts lying. "No scan has run yet" and "we examined forty thousand rows and there is
nothing here" look identical unless someone makes them look different.
So every scan records what each type examined and why it found nothing:
```json
{
"evaluated": [
{
"type": "batch-eligibility",
"rows": 321,
"opportunities": 0,
"reason": "examined 321 rows across the window and found no (model, feature) pair carrying at least 20 non-streaming requests — batch pricing applies to volume that nobody is waiting on, and there is none here at that scale"
},
{
"type": "cache-opportunity",
"rows": 321,
"opportunities": 0,
"reason": "examined 321 rows; 3 model(s) cleared the volume floor and none showed a cacheable repeat population worth acting on — [anthropic/claude-opus-4-8: 0 repeated request(s) across 50 keyable rows (0.0%), below the 10-row floor …]"
}
]
}
```
Read those reasons before concluding you have nothing to save. Each one names the floor it
applied, and a floor you have not cleared yet is a different situation from one you have cleared
and passed.
A scan that returns no recommendations and no reason is refused by the scheduler, so the reason
is structurally always there.
## Turning it on
```yaml
dashboard:
enabled: true
index:
enabled: true # required
analytics:
enabled: true # off by default
recommendations:
enabled: true # the detection loop
interval: 15m
# types: [routing-downgrade, cache-opportunity] # optional allowlist; empty ⇒ all eight
```
The first scan runs at boot; after that it runs on the interval. An unregistered id in `types`
is a named error rather than a silently ignored line, because a detector you believe is running
and is not is worse than a config that will not load.
## What can go wrong
**An applied recommendation is back to `open` after a restart.** Applied state is held in memory
on this build. The audit row and the config change both survive; the card's status does not.
**A card's dollar is missing and you assume nothing is there.** An `UNPROVABLE` projection is not
the absence of an opportunity. `top_tier_share: 0.99` was a solid finding on a card that
rendered no dollar at all.
**The list is empty right after boot.** The first scan runs at startup, but a scan over an empty
or very short ledger finds nothing and says which floor it fell short of.
Detection reads the derived index rather than the log, so the loop's cost is a query rather than
a file scan, and it runs entirely off the request path. Each scan window and each artifact's
scope travel on the card as filter text, so any figure it quotes can be re-derived from your own
rows without trusting the card.