# The embedder ladder

The four ways a semantic rule can turn text into a vector, what each costs per lookup, and which ones make the cache call out.

Source: https://tokentriage.com/docs/build/caching-embedders

---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';

Exact and normalized caching need no embedder. Semantic caching needs one, and choosing it is the
one place in the cache where you trade an air-gapped default for recall quality, latency or money.
Each rung trades a single property, and you pick the rung rather than inheriting its cost.

Embedders are named instances. A rule names one; several rules can share one.

```yaml
cache:
  embedders:
    potion: { kind: static }
  rules:
    - id: chat-hybrid
      strategy: [exact, normalized, semantic]
      semantic:
        embedder: potion
```

## The ladder

| `kind` | Runs | Cost per lookup | Calls out |
|---|---|---|---|
| `lexical` | in process, pure Go | CPU, microseconds | no |
| `static` | in process, pure Go | CPU, microseconds to milliseconds | no |
| `http` at a loopback or local address | your host | a network round trip | yes, gated |
| `http` at an external address | a vendor | a round trip and real spend | yes, gated |

The two in-process kinds are what the default build ships with, and a deployment using them dials
nothing at all.

### `lexical`

Sparse similarity with no model: min-hash and sim-hash signatures over term frequency. High
precision, low recall, and surface-sensitive in a way that turns out to be useful — it notices the
negation, polarity and date differences that dense embeddings smooth over. When it is wrong it is
usually wrong toward a miss.

```yaml
    lex: { kind: lexical }
```

Reach for it when you want a paraphrase tier that will not serve a near-miss, and you can accept
catching fewer paraphrases.

### `static`

A token-to-vector table looked up per token and mean-pooled with weights. No transformer pass, so
it costs a table lookup per token rather than a forward pass, and it runs in process with no
network.

```yaml
    potion:
      kind: static
      table_file: /etc/tokentriage/potion-base-8M.tt
```

<Callout type="warning">
Omitting `table_file` uses the table compiled into the binary, which is 26 tokens across 16
dimensions and exists so tests and the demo path can run offline. Under it `reset` and `change` are
the same vector. Convert a real table offline with the `embedconv` tool and point `table_file` at
it before you promote a semantic rule. TokenTriage never downloads a table.
</Callout>

### `http`

An OpenAI-compatible `/embeddings` client, serving both the local-network rung (a model you host)
and the external rung (a vendor API). The distinction is the address, and both are refused unless
the instance carries `external: allow`.

```yaml
    local:
      kind: http
      base_url: "http://127.0.0.1:11434/v1"    # a model you run
      model: nomic-embed-text
      external: allow
    vendor:
      kind: http
      base_url: "https://api.openai.com/v1"
      model: text-embedding-3-small
      api_key_env: OPENAI_API_KEY              # names a variable, never the secret
      external: allow
```

The external rung is metered: the provider reports token usage and it is priced through the same
path as every other dollar in the product.

## The gate, and what it means

Any embedder that leaves the process is refused at load unless it says so:

```text
cache.embedders["vendor"].external: an external-class embedder (kind http / non-loopback base_url) requires external: allow
```

Opening the gate is one auditable line, and it produces a warning naming the instance and its class:

```text
warning: cache.embedders["vendor"]: external_api embedder — enabling it makes the cache PHONE OUT for embeddings (disclosed on every semantic trace; the air-gap posture is broken for this embedder)
```

An `api_key_env` value must name an environment variable. A literal secret in that field is a
validation error.

## What a metered embedder costs, and where you see it

An embedding runs on every lookup that reaches the semantic link, whether or not it produces a hit,
because the vector is what the search needs. On a miss it runs once: the store write reuses the
vector the lookup already computed rather than embedding a second time.

A priced call lands on the row as `embed_cost`, alongside `embed_us`:

```json
"embed_cost": { "usd": 0.0000021, "estimated": false, "unpriced": false },
"embed_us": 412
```

It is `estimated: false` because it is a real charge against reported usage, not a counterfactual.
If the price does not resolve, the field records `unpriced: true` and makes no dollar claim rather
than reporting zero.

The same spend joins the metacost surface under a `response_cache` line, which is where the cost of
measurement is collected across the whole product.
[What the measurement itself costs](/docs/build/cost-metacost) owns that surface; this page owns
where the number comes from.

An in-process embedder produces no `embed_cost` field at all. It is counted as free because it is:
CPU time on a process you are already running.

<Details summary="One embed per miss, and the exception">

The vector computed during a lookup is carried to the store write, so a metered miss pays for one
embedding rather than two. The exception is a write with no lookup vector — a forced refresh, which
bypasses the read. That embedding has no request row in flight to land on, so it is charged to the
metacost line and counted per rule on the evidence surface instead of being dropped.

</Details>

## Changing an embedder repartitions the cache

The embedder's fingerprint is part of the scope key for a semantic matcher. Swapping the instance a
rule names, or pointing `table_file` at a different table, makes every existing semantic entry
unreachable. Vectors from two different embedders are not comparable, so this is the outcome you
want; plan for a cold semantic tier after the change.

Embedder definitions themselves reload hot, but the effect is the same partition change.

## What can go wrong

**Recall is poor and every paraphrase misses.** Almost always the bundled fixture table. Check
whether `table_file` is set.

**The semantic tier is slower than expected on a large working set.** Embedding is only part of the
cost; the search over a large per-scope set is the rest. See
[budgets and performance](/docs/build/caching-performance).

**A vendor outage turns into misses, not errors.** An embedding failure is a fault that forwards the
request live, and the row records it. The client is served either way.

<NextSteps items={[
  {title: "When a semantic hit is wrong", href: "/docs/build/caching-semantic", description: "What the embedder choice does and does not fix about wrong hits."},
  {title: "What the measurement itself costs", href: "/docs/build/cost-metacost", description: "Where embedding spend joins the rest of the cost of measuring."},
  {title: "The air-gap consent gate", href: "/docs/operate/security#the-air-gap-consent-gate", description: "Every gate that lets this product reach a network service, and what opening one records."}
]} />