Search

↑↓ to move↵ to openesc to closetop 6 documentation results

Book a demoTry TokenTriage

In development · Explore the product preview. Join the launch list or book a walkthrough.

Browse docs

Confidence & the one rule

The five bases a guardrail finding can carry, when it gets a confidence interval, and when it gets none.

ExplanationLast updated

On this page

Every fire may carry a confidence envelope. Its basis says what kind of claim the number is — and four of the five bases can never carry an interval at all.

Basis What it means May carry an interval?
exact a deterministic validator passed; point is 1.0 by construction no
fixture this detector version’s measured precision on a seeded corpus no
calibrated a calibration artifact licenses the bound yes — and only this one
reported the detector’s raw score, explicitly unlicensed no
insufficient a refusal: no number at all no
The five confidence bases and the interval ruleA guardrail finding carries a confidence envelope whose basis comes from a closed set of five. Exact means a deterministic validator passed. Fixture means this detector version's measured precision on a seeded corpus. Reported means the detector's raw score, explicitly unlicensed. Insufficient is the honest refusal to state a number at all. Only the fifth basis, calibrated, may carry an interval, and only while a calibration artifact licenses it: an interval exists if and only if an artifact id sits beside it. The rule is enforced in both directions at the write boundary and again on every read surface, so an envelope that violates it is refused and counted in the row's confidence_refused field rather than being downgraded into a bare number.never an intervalexact — a validator passedfixture — a seeded corpusreported — raw, unlicensedinsufficient — no numbercalibratedthe only basis that may carrya [lo, hi] intervalcalibration artifactqcal_… id on the row⟺confidence_refusedAn envelope that breaks the rule isREFUSED and COUNTED — neverdowngraded into a bare number.a first-class outcome, not an errorartifact missing,stale or adversarialNo interval is EVER rendered for the adversarial category (injection), whatever licenses it —a bound fit on yesterday's attacks does not bound tomorrow's.

The one rule

That counting is the point. Because the refusal is recorded in confidence_refused on the row, “this detector reported no confidence” and “this layer declined to record the confidence it reported” are different facts — on the row and on the page. A system that silently dropped the interval and kept the point would erase that distinction, and the erased version reads as the more confident one.

exact does not mean “certainly a card”

The same distinction applies to media.inventory, whose exact basis means the parse found a part at a known path — a deterministic read of the request bytes, saying nothing whatever about what the part contains.

A calibrated bound changes what threshold means

This is a behavior change that arrives from an artifact activation rather than a config edit, so it is worth stating plainly.

A finding’s score is the licensed point when one exists, in preference to the raw signal. A calibrated point is a version-level aggregate, so while a bound is licensed every finding of that detector version carries the same number. Two consequences:

  • threshold becomes an on/off switch — any value at or below the licensed point admits every finding, any value above it admits none.
  • severity_bands cannot separate a strong hit from a weak one, because there is no such distinction left to separate. A low/medium/high ladder collapses to one rung.

And the calibrated point is not a per-finding probability. It is the version-level share of this detector’s findings adjudicated true positive on one segment and one endpoint — constant across every finding the version produces, and never averaged across coordinates the detector cannot observe.

Four detectors never spend a calibrated bound at all

pii.card, pii.crypto, pii.iban and pii.mac carry only exact rules — a checksum, a Luhn digit, or a fixed structural form — so no calibration artifact is ever consulted for them, on either lane. Their threshold and severity_bands always read the raw signal and mean the same thing buffered or streamed.

No interval is ever rendered for an adversarial category

This is a rule, not a gap awaiting work. injection.heuristic and injection.judge are permanently reported, and three independent code paths refuse an interval there.

insufficient is a first-class outcome

A page that shows insufficient is working correctly. It is not an error state, not a fallback, and not a 0.0 in disguise — it is the layer declining to state a number it does not have evidence for.

localhost:9090/ui/honestySynthetic traffic
One guardrail evidence lineage for code.patterns: 163 fires over 163 rows, all annotated and all from an irreversible action, a reported basis chip with 163 fires and an exemplar of 0.95, and zero adjudicated findings.One guardrail evidence lineage for code.patterns: 163 fires over 163 rows, all annotated and all from an irreversible action, a reported basis chip with 163 fires and an exemplar of 0.95, and zero adjudicated findings.
One lineage on the evidence surface. The basis chip is the census — a count per basis plus one exemplar envelope a detector really produced. It carries no interval, because `reported` may never carry one; and the fires say nothing about whether they were correct.

The same discipline governs the measured figures the detectors themselves publish: below a denominator of 20, the harness refuses to render a rate as a decimal, and prints a tally instead. pii.ssn’s recall rests on 10 positives; pii.iban’s and pii.phone’s on 6. A perfect tally on a small corpus is the most quotable and the least honest number this layer could emit, which is exactly why it is not rendered as one.