# The read model behind the dashboard

A SQLite file derived from your ledger, so the dashboard can answer a query in milliseconds. Why you can delete it, and when you should.

Source: https://tokentriage.com/docs/build/dashboard-read-model

---
import Callout from '../../../components/Callout.astro';
import Details from '../../../components/Details.astro';
import NextSteps from '../../../components/NextSteps.astro';

The dashboard groups a month of spend by tenant in the time it takes to render a chart, over a
ledger that is a JSON Lines file. It does that by keeping a second copy of the interesting
columns in SQLite, and tailing your decision log to keep it current.

Knowing that this file exists changes two operational decisions: what you back up, and what
you do when a surface looks wrong.

## Derived, and disposable

The decision ledger is the system of record. Every request appends one line to it, and that
line is the durable artifact everything else is computed from. The read model is a projection
of those lines into a queryable shape.

Which means the read model has no information the ledger lacks. Delete it and you lose
nothing but the time it takes to rebuild. Corrupt it and the fix is to delete it. Restore a
backup without it and the instance rebuilds it on the next boot.

<Callout type="key">
Back up the ledger and the dashboard state directory. Do not bother backing up the read model:
it is derived from the ledger, and restoring a stale copy is worse than restoring none.
</Callout>

[Backup and restore](/docs/operate/backup-restore) is the executed procedure, and it makes the
same distinction with a worked example.

## Where it lives, and why the path matters

```yaml
dashboard:
  index:
    path: /var/lib/tokentriage/index/decisions.sqlite   # default: ~/.tokentriage/index/decisions.sqlite
```

The default is a single path under your home directory. That is right for one instance on a
laptop and wrong for anything else, because every instance that does not set a path opens the
same file.

Give each instance its own. A shared read model produces two kinds of trouble: contention
between processes, and a boot that spends a long time opening and catching up an index built
from a different ledger. If a start hangs after the replica-identity log line and before the
banner, the index path is where to look.

<Callout type="warning">
`dashboard.index.enabled` decodes but does not disable anything. The read model is opened
whenever the dashboard is composed, and an instance started with the flag set to `false` still
creates the SQLite file and still advertises the read model as available. Treat the read model
as always on when the dashboard is on, and use the path rather than the flag. This is filed.
</Callout>

## Rebuilding it

`index rebuild` deletes the file and replays the whole decision log into a fresh one. It is
deterministic: rebuilding twice from the same log gives the same contents.

```console
$ tokentriage index rebuild --index /var/lib/tokentriage/index/decisions.sqlite \
    --log /var/lib/tokentriage/decisions.jsonl
rebuilt index D:/tt-tmp/ttdocs/state/index/decisions.sqlite from D:/tt-tmp/ttdocs/state/decisions.jsonl
  rows indexed:    0
  indexed_through: file= offset=0 ts=
```

That capture is from an instance whose ledger was empty, which is why the counts are zero and
the position is blank; a populated rebuild reports the row count and the file and offset it
reached. The paths in the output are the ones the command was given.

Run it without arguments to rebuild the defaults, or name both files as above.

**Stop the instance first.** A running instance holds the file open, and on Windows the
rebuild refuses rather than corrupting anything:

```console
$ tokentriage index rebuild
Error: index rebuild: index: remove old index: remove …/decisions.sqlite: The process cannot access the file because it is being used by another process.
```

The instance self-heals on the next boot regardless, so an unplanned restart is a valid
alternative to running the command at all.

## When to rebuild

Four situations, and only two of them are problems.

**A schema change between versions.** Handled for you: the index rebuilds itself when it
opens an older schema. You will see it take longer to boot once after an upgrade.

**After restoring a ledger.** A restored ledger with no index gets one built at boot. A
restored ledger with somebody else's index needs a rebuild, and this is the case where getting
it wrong is quietly wrong rather than loudly wrong.

**A surface disagrees with the CLI.** Both read the same ledger, so a persistent disagreement
means the index is behind or damaged. Rebuild and compare again.

**Disk pressure.** The read model is a second copy, so it costs real space. If it has grown
out of proportion to the ledger it is derived from, delete it and rebuild; nothing in it is
irreplaceable.

<Details summary="What is in it, and what is not">

The index holds the projected columns the dashboard queries on: identifiers, the routing
decision, usage and cost with its pricing snapshot, latency, and the business dimensions you
tagged. It is built by a pure-Go SQLite driver, so the binary stays free of cgo and the
single-file property holds.

It does not hold captured request or response bodies, which live in their own store under
`dashboard.capture.path`. It does not hold your users, tokens, saved views or audit log,
which live in the state directory. And it is not the query engine's only source: a query can
fall back to scanning the canonical log, which is what lets a surface answer while an index is
still catching up.

The provenance envelope on every query response carries an `indexed_through` position — the
file and offset the index has reached. A surface reporting a position behind the end of your
log is telling you it is catching up rather than that rows are missing.

</Details>

## What can go wrong

**The dashboard is behind the CLI.** Read `indexed_through` on a query response. Catching up is
normal after a restart or a large import; a position that stops advancing is not.

**A rebuild fails with a file-in-use error.** Stop the instance. See above.

**Two instances fight over one file.** Both are using the default path. Give each its own.

**The file is far larger than the ledger.** Delete it and rebuild. If the rebuilt file is much
smaller than the one you deleted, the old one had accumulated space it never gave back, and the
rebuild is the reclaim.

<NextSteps items={[
  {title: "The decision ledger on disk", href: "/docs/operate/ledger-lifecycle", description: "The file this one is derived from: how it grows, rotates and is pruned."},
  {title: "Back up and restore", href: "/docs/operate/backup-restore", description: "An executed procedure that separates the system of record from the derived files."},
  {title: "Turn on the dashboard", href: "/docs/build/dashboard-enable", description: "The dashboard block, including the index path this page argues you should set."},
  {title: "Diagnose a running instance", href: "/docs/operate/diagnostics", description: "Reading index health and row counts without stopping anything."}
]} />