Secrets & injection detectors
The three secrets detectors and what each costs in precision and microseconds, plus the injection matcher and its recall floor.
On this page
Six registrations across two categories. The nine PII detectors are on their own page, the phrase and content matchers on the next one, and the inventory has the summary table.
secrets — three separate registrations, deliberately
The v2 split removed assignment-shaped secrets from secrets.ruleset (now v3): upstream’s single
generic-api-key rule moved to secrets.generic, a separate opt-in. The split did not make the
capability cheaper — it made declining it possible.
The following are the published, authored fixture observations from the original detector split, not a blinded holdout or a current population estimate. Detector versions and scanning bounds have since changed; these historical figures do not qualify the current release.
| detector | covers | historical fixture observations, with denominators |
|---|---|---|
secrets.ruleset |
220 ported Gitleaks provider rules — AWS, GitHub, GitLab, Slack, Stripe, Google, Azure, npm, PyPI, SendGrid, Twilio and ~200 more — by documented key shape; PEM blocks and structurally valid JWTs confirmed by a deterministic structure check | recall 0.26 (10 of 39); precision 10 of 10 findings — a tally, not a rate; 0 false positives on both negative sets |
secrets.generic |
assignment-context credentials with no recognizable issuer (api_key = …, password: …, token => …), producing the API_KEY_ASSIGNMENT / PASSWORD_ASSIGNMENT / GENERIC_TOKEN / GENERIC_SECRET labels nothing else in the tree produces |
recall contribution 0.08 (3 of 39); precision 3 of 6 findings; it is the sole source of all 3 of the secrets fleet’s false positives |
secrets.entropy |
quoted or keyword-assigned strings clearing verified Shannon thresholds (base64 ≥ 4.5 bits/char, hex ≥ 3) | recall 0.10 (4 of 39); precision 4 of 8 findings; 0.0198 (2/101) hard negatives and 0.0238 (1/42) clean controls |
Shared gaps across the three. Credentials are not checked with an issuer for liveness or
revocation. Values split across separate message fields are not reassembled. Scans are bounded
to 128 KiB per field and per document total, and 256 findings per field; reaching a scanning cap reports scan_exhausted
rather than a clean complete scan. The findings cap also emits SECRETS_SCAN_TRUNCATED.
The former silent first-32-base64-run cutoff has been removed; candidate walking remains bounded,
and exhaustion is disclosed. These bounds do not guarantee every credential will be found.
Upstream’s own allowlists are ported faithfully, and they suppress real positives. AWS keys
ending in EXAMPLE account for 5 of secrets.ruleset’s 29 corpus misses in one stroke; per-rule
entropy bars silence named issuers completely rather than partially (Slack catches 0 of 5). For
secrets.generic, disabling the allowlists in a counterfactual arm takes recall from 0.08 to
0.26 — 7 more true positives. That is not an argument for deleting the list: the corpus’s own
provenance caveat says its positives are drawn from published example values real rulesets
allowlist, so the figure measures port fidelity more than field generalization.
injection.heuristic — four phrase families
Covers. Ignore-previous-instructions (an imperative aimed at a scope word and an instruction object, within one sentence); role-override / persona-replacement (“you are no longer…”, “act as an unrestricted…”, “developer|god|admin|root mode”); system-prompt extraction (“repeat the text above”, a disclose verb aimed at the model’s own prompt); and delimiter escape (forged role tags, ChatML and Llama control tokens, a markdown heading declaring a system block). Each family survives zero-width and homoglyph disguise, and each carried base64-encoded when the run is ≥ 12 characters — with the span over the encoded bytes, so redacting it removes the payload.
Not covered.
- This is a low-recall floor and it recognizes attacks somebody already wrote down. A paraphrase outside the four families is a miss, and enumerating more phrasings does not change that — prompt injection is an open natural-language problem and a phrase matcher is its weakest rung. A deployment that binds only this detector has a keyword filter, not injection coverage.
- A directive quoted inside a mention frame is refused — a match sitting entirely inside a quotation introduced by one of ~30 mention phrases (“phrases such as”, “for example”, “the string”) is dropped as documentation. An attacker who quotes their payload and prefixes it with a mention phrase defeats this detector. The cost and the benefit are both measured every build.
- Every pattern is English-only, and that is a coverage boundary rather than an oversight. Semantic, narrative, translated and multi-turn attacks carry no family phrasing and are misses.
- Indirect injection via a resource this layer never sees — a URL the model is asked to fetch, a tool result, a retrieved document arriving outside the inspected request fields.
- The split-token squeeze projection is OFF, and that is an ablation result rather than an
omission. It cost ~45% of the detector’s runtime, caught zero additional corpus positives, and
caused a false positive. So a directive broken so thoroughly that no rule anchor survives —
ig nore all pre vious inst ructions— is a miss. - Documents over 128 KiB are refused and disclosed as
detector_error, not truncated. - A per-finding probability of any kind. Every finding carries
basis: reportedand never an interval. See the adversarial rule below.
The external tiers
Stock sidecar activation is rejected pending an independently qualified pinned runtime and
model. The three sidecar.* registrations describe intended classifier tasks; they are not
proof of NER spans, supported languages or detection efficacy in this release.
injection.judge is an input-only LLM classifier. Its boolean verdict produces a finding at
score 1.0 with basis: reported, or no finding for a clean verdict. There is no score gradient or
calibrated interval. Live-model precision, recall and susceptibility to attacks on the judge are
unmeasured. Synthetic graders and actual local HTTP transports test protocol, cancellation,
accounting and persisted receipts; they do not establish model quality.
- Consent and budgets are enforced. The selected destination needs explicit consent. Calls share a request-local reservation/settlement budget. Stock activation with a positive hard dollar cap is rejected until the configured pricer proves a conservative reservation bound. An explicit zero cap sends zero calls. Uncapped calls may proceed with explicitly unpriced cost.
- Call receipts are independent of findings. Clean, hit, failed, cancelled and preflight-denied attempts remain visible through execution and invocation receipts. Model/destination identities are hashed; known usage is priced even when the verdict is malformed. A sent attempt does not prove remote receipt, and absent usage does not mean a free call.
- The judge model’s license is unknown in the descriptor. There is no judge
model_licenseconfiguration field in this build. Sidecar operator declarations are separate metadata, not verified license compliance. No model weights are bundled. - Input only, bounded to 32 KiB of inspected text. Oversize input refuses the call with
detector_error; timeouts and cancellation are disclosed. Continuation follows the effective mode/posture, and cancellation may occur after a send. - Post-sanitization blindness is deliberate. Local privacy transformations precede eligible
external work. A value successfully removed by a local guard cannot be inspected by the judge.
Incomplete enforcing sanitation withholds secondary content even when the primary request’s
open posture allows continuation. Stock external
NeedsRawand spanless mutation bindings are rejected;raw_egress_ackdoes not authorize them.
| sidecar registration | intended classifier task; stock activation unavailable |
|---|---|
sidecar.ner_pii |
free-text PII classification; no qualified NER span implementation or efficacy claim |
sidecar.safety |
harmful-content classification; no qualified generative safety adapter or language coverage claim |
sidecar.injection |
semantic injection classification; actual model efficacy remains unmeasured |
The reference service is a development fixture. A reachable service or passing protocol test is not evidence of a qualified model deployment. See Air-gap & limits.