Testing instrumented code

Instrumentation rots quietly. Somebody refactors a call out of its step block, the labels stop being set, and nothing fails — you find out a month later when a dashboard cannot answer a question it used to answer.

The fix is a test that asserts your calls carry the labels you expect. The SDK ships a small kit for exactly that, and it needs no proxy, no network and no fixtures of your own.

What you get

Two pieces per language, and they mirror each other.

Both are optional modules. Python’s needs httpx installed, since it builds on that transport interface; TypeScript’s is built in.

“Final headers” is the important part: what is captured is the map after the adapter has injected, which is what actually would have gone on the wire.

A test that catches the refactor

import tokentriage as tt
from tokentriage.testing import RecordingTransport, assert_tt_headers

def test_my_calls_are_attributed():
    tt.init(base_url="http://tt.local")
    transport = RecordingTransport(response_headers={"x-tt-request-id": "tt_test"})
    client = tt.httpx_client(transport=transport, base_url="http://tt.local")

    with tt.trace(agent="my-agent", user="u-1"):
        with tt.step(step_class="user_turn"):
            resp = client.post("http://tt.local/v1/chat/completions", json={})
            rcpt = tt.receipt(resp)

    assert_tt_headers(
        transport.last_request_headers,
        {"x-tt-agent": "my-agent", "x-tt-user": "u-1", "x-tt-step-class": "user_turn"},
    )
    assert rcpt.request_id == "tt_test"

import * as tt from "tokentriage";
const { recordingFetch, assertTtHeaders } = tt.testing;

test("my calls are attributed", async () => {
  tt.init({ baseUrl: "http://tt.local" });
  const rec = recordingFetch({ responseHeaders: { "x-tt-request-id": "tt_test" } });
  const f = tt.wrapFetch(rec, { baseUrl: "http://tt.local" });

  await tt.trace({ agent: "my-agent", user: "u-1" }, async () => {
    await tt.step({ stepClass: "user_turn" }, async () => {
      const r = await f("http://tt.local/v1/chat/completions", { method: "POST", body: "{}" });
      expect(tt.receipt(r).requestId).toBe("tt_test");
    });
  });

  assertTtHeaders(rec.requests[0].headers, {
    "x-tt-agent": "my-agent",
    "x-tt-user": "u-1",
  });
});

http://tt.local does not resolve and nothing tries to reach it. The captured headers are what the adapter built:

x-tt-trace-id  = bbbf47dfd02974f1483f8759f5598148
x-tt-step-id   = e0aad7fe4d96b2f7
x-tt-agent     = my-agent
x-tt-step-class= user_turn
x-tt-user      = u-1

Move the call outside the step block and the assertion on x-tt-step-class fails, which is the whole point.

When it fails

The diff names the header, both values, and what was present, so you do not have to print the map yourself:

assert_tt_headers mismatch:
  'x-tt-agent': expected 'wrong-agent', got 'my-agent'
  x-tt-* headers present: x-tt-agent, x-tt-step-class, x-tt-step-id, x-tt-trace-id, x-tt-user

TypeScript prints the same shape and throws an error named AssertionError, matching what Python raises, so a shared CI log reads the same either way.

Matching is case-insensitive. Both take an absent argument for the opposite assertion — that a header is not there, which is how you test that off-origin requests stay clean.

Canned responses

The recorder returns a TokenTriage-shaped response you control, so you can test your handling of states that are awkward to produce live:

RecordingTransport(
    response_headers={"x-tt-request-id": "tt_x", "x-tt-cache": "hit",
                      "x-tt-cache-kind": "semantic", "x-tt-cache-verified": "none"},
    status=200,
)

recordingFetch({
  responseHeaders: { "x-tt-request-id": "tt_x", "x-tt-cache": "hit",
                     "x-tt-cache-kind": "semantic", "x-tt-cache-verified": "none" },
  status: 200,
});

A 429 with refusal headers, an unverified semantic cache hit, a response with no request id at all — each is a header dict away, and each exercises a branch of your code that is otherwise hard to reach.

What it will not fabricate

The kit builds wire shapes. It does not build money.

There is no way to construct a PRICED cost in either language, in a test or anywhere else, because Cost has no public constructor and the only producers are the receipt parser and the ledger row mapper over server JSON. If you want to test your handling of a priced row, feed your code a row from a real fetch, or test the branch through render output rather than by minting a cost object.

That is a deliberate limit rather than a missing feature. A test fixture that can mint a dollar is a path by which a dollar can be minted.