Translated streams

Your stream delivered two chunks and stopped. The connection stayed up, no terminator arrived, and the last thing on the wire was an object with an error key in it. Your SDK may have raised that as an exception, or it may have handed you a short answer and moved on.

That is a named case rather than a mystery. A frame from the upstream could not be re-encoded into the format your client speaks, so the exchange ended there. The content that already reached you is yours to keep, and what happened is written on the decision-log row for the request, carrying the same error string you saw.

This page is about the middle of a translated stream going wrong: what your client gets, what the row records, and the one configuration pair that will not load at all.

Read Which client formats reach which upstreams first if you are not sure your pair streams in the first place. The disclosure block itself is covered field by field in What a translation drops, and how you find out; this page uses it rather than re-teaching it.

The stream that stopped early

One thing splits the diagnosis, and it is the last frame your client read off the socket.

The last thing on the wire	What it means	Where to look next
data: [DONE], or message_stop	the stream finished	the row’s outcome.finish_reason reads stop
a data: line carrying an error object	a frame could not be re-encoded, and the exchange is over	the row’s outcome.error, which carries the same string
neither, and the socket closed	a different problem from this one	When a provider won’t connect

Here is that middle case, captured. An OpenAI-format client streams POST /v1/chat/completions; the route sends it to an Anthropic-dialect upstream; after two good frames the upstream emits a content_block_delta event whose body is not valid JSON.

=== RUN   TestStreamTranslationMidFailure
    translate_stream_test.go:382: client transcript after the poison frame:
        data: {"choices":[{"delta":{"role":"assistant"},"finish_reason":null,"index":0}],"created":1787629035,"id":"msg_mock_0000000000","model":"claude-haiku-4-5","object":"chat.completion.chunk"}

        data: {"choices":[{"delta":{"content":"Hello"},"finish_reason":null,"index":0}],"created":1787629035,"id":"msg_mock_0000000000","model":"claude-haiku-4-5","object":"chat.completion.chunk"}

        data: {"error":{"message":"translate: anthropic stream frame \"content_block_delta\" is not valid JSON","type":"gateway_error"}}

That is one run of go test ./test/e2e/ -run '^TestStreamTranslationMidFailure$' -v -count=1 in the product repository, 2026-08-24, against a mock upstream that injects the bad frame on demand. The created value and the msg_mock_… id come from the mock and yours will differ; the frame shapes and the error string are structural, and the string is what an operator greps for. The two lines that reached the client before the failure are the client’s: partial content is not withdrawn.

What your client receives

An error event is terminal, and the encoder that writes it does not go on to write the format’s success terminator. An OpenAI-format client therefore gets no data: [DONE], and an Anthropic-format client gets no message_stop. A client watching only for the success terminator still sees the stream end abnormally.

An OpenAI-format client gets one SSE data: line carrying an OpenAI error envelope, the same envelope a buffered failure would produce:

data: {"error":{"message":"translate: anthropic stream frame \"content_block_delta\" is not valid JSON","type":"gateway_error"}}

That line is copied from the capture above, so its conditions are the conditions of that run. The type value gateway_error is the gateway’s own: naming an upstream error type that no upstream sent would misattribute the failure. Where the failure carries no detail of its own, the message is the literal stream translation failed.

An Anthropic-format client gets a named frame instead: an event: error line, then a data: line whose object carries "type":"error" at the top level and an inner error object whose own type is api_error. Both encoders build that envelope from the registered error adapter rather than by hand, which is why a streamed failure and a buffered one arrive in the same shape.

Two further upstreams behind the same OpenAI-format client behave the same way, with the message naming whichever decoder gave up: TestStreamTranslationMidFailureGemini and TestStreamTranslationMidFailureBedrock are the tests you can run for those.

What the row records

Four blocks lifted out of the single row that same run logged, wrapped here for width. The routing, trace and latency blocks are omitted, as are the would_usd, signals and oracle sub-objects inside these four. Every value shown is as captured.

"usage": {"input_tokens":200,"output_tokens":0,"cache_read_tokens":64,"cache_write_tokens":16,
          "reasoning_tokens":0,"total_tokens":200,"source":"provider"},
"cost": {"usd":0.0002264,"pricing_key":"anthropic/claude-haiku-4-5",
         "pricing_snapshot":"litellm@2026-07-14#179affb","estimated":false},
"outcome": {"finish_reason":"","error":"translate: anthropic stream frame \"content_block_delta\" is not valid JSON",
            "client_disconnect":false},
"disclosures": {"translation":{"from":"openai","to":"anthropic","approximations":["max_tokens:defaulted-to-4096"]}}

Read it in three passes.

finish_reason is empty, and it is not stop. A truncated stream is not dressed up as a completed one. Any aggregate that counts completions by that field leaves this request out.

outcome.error carries the string your client saw. The failure is on the receipt as well as on the wire, so an operator who was not holding the socket can still find the request afterwards by grepping for the same text.

The partial work is still priced. 200 input tokens, zero output tokens, and $0.0002264 under anthropic/claude-haiku-4-5. The two frames that did relay are not written off, and the output tokens that never arrived are not invented. Those token counts come from a mock upstream, so the dollar is arithmetic you can check rather than anyone’s bill.

The disclosure block is present, and shorter than it would have been on a clean run of the same route, because the stream ended before any frame that would have been dropped came through. It reports what happened to this request.

Why a mid-stream discovery still reaches the receipt

Non-streaming translation buffers the upstream body and re-renders it once. A stream has no body to buffer, so the same lane runs two event codecs instead, the upstream’s decoder and your format’s encoder: each frame becomes a canonical event and is written out again in your shape, one at a time, so the pacing survives.

Anything that will not re-encode goes into a set that is read once, at stream close, and folded into the same disclosures.translation block the request-side translation already wrote. None of it travels in a header. Response headers went out long before the third frame existed, and the row is not built until the tap closes, so a discovery made in the middle of a stream lands on the receipt like any other.

Here is a clean run of the same pair, where the upstream sends Anthropic’s extended-thinking shape:

    translate_stream_test.go:769: row loss: dropped=[stream.anthropic.delta:signature_delta stream.thinking_delta:unrepresentable-in-openai] approximations=[max_tokens:defaulted-to-4096]

That is from go test ./test/e2e/ -run '^TestStreamTranslationLossOnTheRow$' -v -count=1, same repository and date, with the upstream fixture taken from Anthropic’s own published streaming shape. The client got the answer text and an ordinary data: [DONE]. Two things did not survive, and the two entries are lost on opposite sides of the crossing: an OpenAI chunk has no reasoning-bearing field for the thinking delta to land in, and no canonical event carries a thinking block’s integrity signature in the first place. Both are named on the row. Truncating either quietly is what the disclosure mechanism exists to rule out.

The set deduplicates and is bounded at sixteen distinct entries. Past that it appends stream.loss_inventory:truncated rather than growing without limit, so a pathological stream cannot inflate a row.

Which bytes the money comes from

A translated streaming row prices from the upstream’s own frames, in the upstream’s own dialect. A translated non-streaming row prices from the response body after it has been re-rendered into your dialect. Same feature, two provenances — and a reader reconciling two rows needs to know which story each is telling.

The difference is the buffering. On the non-streaming path the body is re-rendered before the ledger tap sees it, which puts the obligation on the response adapter to re-encode the upstream’s token counts faithfully. On the streaming path there is no re-rendered body to read, so the ledger scanner is built from the translation target instead of from the client.

One consequence lands on a specific pair, and it is disclosed rather than hidden. An Anthropic-format client streaming from an OpenAI upstream sees zeros for usage in its own stream: OpenAI emits its usage chunk after the finish chunk, while Anthropic carries usage on message_delta, which the encoder has to write at finish time. Two receipts ride the row and say so, stream.anthropic.message_start.usage:not-yet-known→0 and stream.anthropic.message_delta.usage:absent→0, and the row itself is priced from the upstream’s numbers. If you need per-request usage from an Anthropic-format client, read the row.

Stream-phase guardrails and translated streams will not load together

Arm the guardrail layer, give any guard the stream phase, and set routing.allow_cross_dialect_streaming: true, and the process does not start. Validation says so first, and it names both offending lines.

$ tokentriage validate --config t7-guard-stream-vs-translated-stream.yaml
Error: configuration is invalid: config D:/tt-w01-neutral/work/t7-guard-stream-vs-translated-stream.yaml: guardrails.guards[0].phases[1]: guard "pii-stream" inspects at the "stream" phase, which is REFUSED while routing.allow_cross_dialect_streaming is true (BB14). The stream-framing gate supports exactly (openai, chat) and is armed from the ROUTE before the first byte relays (internal/proxy/stream_framing.go:34-97,127-129); it assumes streams never cross dialects, so on a translated stream it would arm an inspector over frames it cannot decode and produce a "guarded, clean" row. Turn off one of the two: drop the "stream" phase from this guard, or set routing.allow_cross_dialect_streaming: false

Exit status 1, from a binary built at the same commit as the captures above. The absolute config path inside the message is the capture machine’s and yours will hold your own path; the sentence itself is the artefact. The binary prints this message twice: the command layer prints it, then the top-level handler prints it again, because the root command sets SilenceUsage without setting SilenceErrors. The second copy is elided above. It is a cosmetic defect in validate, and one error reported as two is not a sign that your config carries two.

The gate that decides whether a stream can be inspected supports one shape, an OpenAI chat stream, and it is armed from the route before the first byte relays, because deciding afterwards would mean deciding after content had already reached the client. That gate assumes a stream does not cross a dialect boundary. Turning translated streams on removes the assumption, and an inspector armed over frames it cannot decode would write the one row that must not exist: guarded, clean. Not booting is the best of the three options that were considered. It was chosen over silently skipping guardrails on translated streams, which would leave an undisclosed enforcement hole, and over stopping the stream at runtime, which springs a config-time-knowable conflict on an operator mid request.

The check keys on three facts the config owns: the flag, guardrails.enabled, and any guard carrying a stream phase. Whether a given request would in fact meet both features is not computable from the file, so the check is broader than the conflict it protects against, which is the correct direction for a gate that fails closed. It also means a disarmed guardrail layer holding a stream-phase guard loads clean today and is turned away later, at the flip that arms it.

Either single flip validates clean

Both remedies are one line, and they are genuine alternatives rather than a preference.

# Remedy A - keep stream-phase inspection, turn translated streams off.
routing:
  allow_cross_dialect: true
  allow_cross_dialect_streaming: false

# Remedy B - keep translated streams, take the stream phase off the guard.
guardrails:
  guards:
    - id: pii-stream
      phases: [input]

Remedy A leaves translated buffered traffic working; a streaming request that would have crossed a dialect falls open to the same-dialect default upstream instead. Remedy B leaves the input and output phases inspecting the buffered halves as before. Each was run through the same command against the refused config with one line changed:

$ tokentriage validate --config t7-fix-a-streaming-off.yaml
warning: routing.rules is empty in route mode; only routing.default_tier and overrides.tier_header will apply
mode:      route
listen:    127.0.0.1:8787   admin: 127.0.0.1:9090
upstreams: 2
  - anthropic    anthropic https://api.anthropic.com
  - openai       openai    https://api.openai.com
tiers:     2 (cheapest -> most capable)
  0. cheap      -> anthropic/claude-haiku-4-5
  1. frontier   -> openai/gpt-4o
rules:     0
OK: configuration is valid

Exit status 0. That warning: line is also printed twice by the binary, by a different route from the doubled error, and the repeat is elided here too. Remedy B prints the same summary; the two captures differ only in the filename echoed after --config. The upstream names, ports and tiers belong to the capture config, so yours will read differently.

The follow-up that would let you have both is already named: guardrail framing that understands more than one codec widens the supported set, and this check narrows or folds when that lands.

What this does not do

• Both flags are off out of the box. routing.allow_cross_dialect_streaming requires routing.allow_cross_dialect, and neither defaults to true. With streaming translation off, a streaming request aimed across a dialect boundary goes to the same-dialect default upstream and writes no disclosure block at all.
• A translated stream is not byte-faithful. That is what translating means. Same-dialect streaming stays byte-identical through the translation layer even with translation enabled, and TestSameDialectStreamUntouchedWithTranslationEnabled is the test you can run for it. One other layer does touch a same-dialect stream: on a streamed chat request to an openai-dialect upstream the gateway splices stream_options.include_usage into the outbound body and removes the resulting terminal usage frame from your copy if you did not ask for it, which is how the row carries provider-reported usage instead of an estimate. When a stream misbehaves has the switch and what turning it off costs.
• No upstream frame is relayed to the client, on either outcome; TestTranslatedStreamNeverRelaysUpstreamFrames pins that.
• There is no retry. Headers went out with the first frame, so the exchange is terminal by the time a later frame can go wrong.
• A non-2xx upstream reply is a different case. It relays untranslated, in the served dialect’s error shape, with the status preserved. What this page describes is a 200 that went bad after the headers were committed.
• An error event authored by the upstream is terminal too, and gets no second gateway-authored envelope stacked on top of it. The row then prefers the upstream’s own reason, because that is the root cause and the codec failure is its consequence. If outcome.error reads like something the provider said, the provider said it.
• This page covers chat streams. Which pairs can stream at all, and which two dialects TokenTriage drives outward without accepting inward, belong to the matrix.