# Translated streams What a translated stream does when a frame will not encode: the client's own error shape, a recorded outcome error, and loss at finish. Source: https://tokentriage.com/docs/build/providers-streaming-translation --- import Callout from '../../../components/Callout.astro'; import NextSteps from '../../../components/NextSteps.astro'; Your stream delivered two chunks and stopped. The connection stayed up, no terminator arrived, and the last thing on the wire was an object with an `error` key in it. Your SDK may have raised that as an exception, or it may have handed you a short answer and moved on. That is a named case rather than a mystery. A frame from the upstream could not be re-encoded into the format your client speaks, so the exchange ended there. The content that already reached you is yours to keep, and what happened is written on the decision-log row for the request, carrying the same error string you saw. This page is about the middle of a translated stream going wrong: what your client gets, what the row records, and the one configuration pair that will not load at all. Read [Which client formats reach which upstreams](/docs/build/providers-dialect-matrix) first if you are not sure your pair streams in the first place. The disclosure block itself is covered field by field in [What a translation drops, and how you find out](/docs/build/providers-translation-loss); this page uses it rather than re-teaching it. ## The stream that stopped early One thing splits the diagnosis, and it is the last frame your client read off the socket. | The last thing on the wire | What it means | Where to look next | |---|---|---| | `data: [DONE]`, or `message_stop` | the stream finished | the row's `outcome.finish_reason` reads `stop` | | a `data:` line carrying an `error` object | a frame could not be re-encoded, and the exchange is over | the row's `outcome.error`, which carries the same string | | neither, and the socket closed | a different problem from this one | [When a provider won't connect](/docs/build/providers-troubleshooting) | Here is that middle case, captured. An OpenAI-format client streams `POST /v1/chat/completions`; the route sends it to an Anthropic-dialect upstream; after two good frames the upstream emits a `content_block_delta` event whose body is not valid JSON. ```text === RUN TestStreamTranslationMidFailure translate_stream_test.go:382: client transcript after the poison frame: data: {"choices":[{"delta":{"role":"assistant"},"finish_reason":null,"index":0}],"created":1787629035,"id":"msg_mock_0000000000","model":"claude-haiku-4-5","object":"chat.completion.chunk"} data: {"choices":[{"delta":{"content":"Hello"},"finish_reason":null,"index":0}],"created":1787629035,"id":"msg_mock_0000000000","model":"claude-haiku-4-5","object":"chat.completion.chunk"} data: {"error":{"message":"translate: anthropic stream frame \"content_block_delta\" is not valid JSON","type":"gateway_error"}} ``` That is one run of `go test ./test/e2e/ -run '^TestStreamTranslationMidFailure$' -v -count=1` in the product repository, 2026-08-24, against a mock upstream that injects the bad frame on demand. The `created` value and the `msg_mock_…` id come from the mock and yours will differ; the frame shapes and the error string are structural, and the string is what an operator greps for. The two lines that reached the client before the failure are the client's: partial content is not withdrawn. ## What your client receives An `error` event is terminal, and the encoder that writes it does not go on to write the format's success terminator. An OpenAI-format client therefore gets no `data: [DONE]`, and an Anthropic-format client gets no `message_stop`. A client watching only for the success terminator still sees the stream end abnormally. **An OpenAI-format client** gets one SSE `data:` line carrying an OpenAI error envelope, the same envelope a buffered failure would produce: ```text data: {"error":{"message":"translate: anthropic stream frame \"content_block_delta\" is not valid JSON","type":"gateway_error"}} ``` That line is copied from the capture above, so its conditions are the conditions of that run. The `type` value `gateway_error` is the gateway's own: naming an upstream error type that no upstream sent would misattribute the failure. Where the failure carries no detail of its own, the message is the literal `stream translation failed`. **An Anthropic-format client** gets a named frame instead: an `event: error` line, then a `data:` line whose object carries `"type":"error"` at the top level and an inner `error` object whose own `type` is `api_error`. Both encoders build that envelope from the registered error adapter rather than by hand, which is why a streamed failure and a buffered one arrive in the same shape. **The OpenAI-format frame above is captured. The Anthropic-format one is not.** No end-to-end test drives an Anthropic-format client through a mid-stream failure, so its bytes were read out of the encoder and a unit test over the envelope shape, not off a wire. Treat the block as evidence and the paragraph as a description of a shape nobody has yet watched arrive. Two further upstreams behind the same OpenAI-format client behave the same way, with the message naming whichever decoder gave up: `TestStreamTranslationMidFailureGemini` and `TestStreamTranslationMidFailureBedrock` are the tests you can run for those. ## What the row records Four blocks lifted out of the single row that same run logged, wrapped here for width. The routing, trace and latency blocks are omitted, as are the `would_usd`, `signals` and `oracle` sub-objects inside these four. Every value shown is as captured. ```json "usage": {"input_tokens":200,"output_tokens":0,"cache_read_tokens":64,"cache_write_tokens":16, "reasoning_tokens":0,"total_tokens":200,"source":"provider"}, "cost": {"usd":0.0002264,"pricing_key":"anthropic/claude-haiku-4-5", "pricing_snapshot":"litellm@2026-07-14#179affb","estimated":false}, "outcome": {"finish_reason":"","error":"translate: anthropic stream frame \"content_block_delta\" is not valid JSON", "client_disconnect":false}, "disclosures": {"translation":{"from":"openai","to":"anthropic","approximations":["max_tokens:defaulted-to-4096"]}} ``` Read it in three passes. **`finish_reason` is empty, and it is not `stop`.** A truncated stream is not dressed up as a completed one. Any aggregate that counts completions by that field leaves this request out. **`outcome.error` carries the string your client saw.** The failure is on the receipt as well as on the wire, so an operator who was not holding the socket can still find the request afterwards by grepping for the same text. **The partial work is still priced.** 200 input tokens, zero output tokens, and $0.0002264 under `anthropic/claude-haiku-4-5`. The two frames that did relay are not written off, and the output tokens that never arrived are not invented. Those token counts come from a mock upstream, so the dollar is arithmetic you can check rather than anyone's bill. The disclosure block is present, and shorter than it would have been on a clean run of the same route, because the stream ended before any frame that would have been dropped came through. It reports what happened to this request. ## Why a mid-stream discovery still reaches the receipt Non-streaming translation buffers the upstream body and re-renders it once. A stream has no body to buffer, so the same lane runs two event codecs instead, the upstream's decoder and your format's encoder: each frame becomes a canonical event and is written out again in your shape, one at a time, so the pacing survives. Anything that will not re-encode goes into a set that is read once, at stream close, and folded into the same `disclosures.translation` block the request-side translation already wrote. None of it travels in a header. Response headers went out long before the third frame existed, and the row is not built until the tap closes, so a discovery made in the middle of a stream lands on the receipt like any other. Here is a clean run of the same pair, where the upstream sends Anthropic's extended-thinking shape: ```text translate_stream_test.go:769: row loss: dropped=[stream.anthropic.delta:signature_delta stream.thinking_delta:unrepresentable-in-openai] approximations=[max_tokens:defaulted-to-4096] ``` That is from `go test ./test/e2e/ -run '^TestStreamTranslationLossOnTheRow$' -v -count=1`, same repository and date, with the upstream fixture taken from Anthropic's own published streaming shape. The client got the answer text and an ordinary `data: [DONE]`. Two things did not survive, and the two entries are lost on opposite sides of the crossing: an OpenAI chunk has no reasoning-bearing field for the thinking delta to land in, and no canonical event carries a thinking block's integrity signature in the first place. Both are named on the row. Truncating either quietly is what the disclosure mechanism exists to rule out. The set deduplicates and is bounded at sixteen distinct entries. Past that it appends `stream.loss_inventory:truncated` rather than growing without limit, so a pathological stream cannot inflate a row. ## Which bytes the money comes from A translated **streaming** row prices from the upstream's own frames, in the upstream's own dialect. A translated **non-streaming** row prices from the response body after it has been re-rendered into your dialect. Same feature, two provenances — and a reader reconciling two rows needs to know which story each is telling. The difference is the buffering. On the non-streaming path the body is re-rendered before the ledger tap sees it, which puts the obligation on the response adapter to re-encode the upstream's token counts faithfully. On the streaming path there is no re-rendered body to read, so the ledger scanner is built from the translation target instead of from the client. One consequence lands on a specific pair, and it is disclosed rather than hidden. An Anthropic-format client streaming from an OpenAI upstream sees zeros for usage in its own stream: OpenAI emits its usage chunk after the finish chunk, while Anthropic carries usage on `message_delta`, which the encoder has to write at finish time. Two receipts ride the row and say so, `stream.anthropic.message_start.usage:not-yet-known→0` and `stream.anthropic.message_delta.usage:absent→0`, and the row itself is priced from the upstream's numbers. If you need per-request usage from an Anthropic-format client, read the row. ## Stream-phase guardrails and translated streams will not load together Arm the guardrail layer, give any guard the `stream` phase, and set `routing.allow_cross_dialect_streaming: true`, and the process does not start. Validation says so first, and it names both offending lines. ```console $ tokentriage validate --config t7-guard-stream-vs-translated-stream.yaml Error: configuration is invalid: config D:/tt-w01-neutral/work/t7-guard-stream-vs-translated-stream.yaml: guardrails.guards[0].phases[1]: guard "pii-stream" inspects at the "stream" phase, which is REFUSED while routing.allow_cross_dialect_streaming is true (BB14). The stream-framing gate supports exactly (openai, chat) and is armed from the ROUTE before the first byte relays (internal/proxy/stream_framing.go:34-97,127-129); it assumes streams never cross dialects, so on a translated stream it would arm an inspector over frames it cannot decode and produce a "guarded, clean" row. Turn off one of the two: drop the "stream" phase from this guard, or set routing.allow_cross_dialect_streaming: false ``` Exit status 1, from a binary built at the same commit as the captures above. The absolute config path inside the message is the capture machine's and yours will hold your own path; the sentence itself is the artefact. **The binary prints this message twice**: the command layer prints it, then the top-level handler prints it again, because the root command sets `SilenceUsage` without setting `SilenceErrors`. The second copy is elided above. It is a cosmetic defect in `validate`, and one error reported as two is not a sign that your config carries two. The gate that decides whether a stream can be inspected supports one shape, an OpenAI chat stream, and it is armed from the route before the first byte relays, because deciding afterwards would mean deciding after content had already reached the client. That gate assumes a stream does not cross a dialect boundary. Turning translated streams on removes the assumption, and an inspector armed over frames it cannot decode would write the one row that must not exist: `guarded, clean`. Not booting is the best of the three options that were considered. It was chosen over silently skipping guardrails on translated streams, which would leave an undisclosed enforcement hole, and over stopping the stream at runtime, which springs a config-time-knowable conflict on an operator mid request. The check keys on three facts the config owns: the flag, `guardrails.enabled`, and any guard carrying a `stream` phase. Whether a given request would in fact meet both features is not computable from the file, so the check is broader than the conflict it protects against, which is the correct direction for a gate that fails closed. It also means a disarmed guardrail layer holding a `stream`-phase guard loads clean today and is turned away later, at the flip that arms it. ### Either single flip validates clean Both remedies are one line, and they are genuine alternatives rather than a preference. ```yaml # Remedy A - keep stream-phase inspection, turn translated streams off. routing: allow_cross_dialect: true allow_cross_dialect_streaming: false # Remedy B - keep translated streams, take the stream phase off the guard. guardrails: guards: - id: pii-stream phases: [input] ``` Remedy A leaves translated *buffered* traffic working; a streaming request that would have crossed a dialect falls open to the same-dialect default upstream instead. Remedy B leaves the `input` and `output` phases inspecting the buffered halves as before. Each was run through the same command against the refused config with one line changed: ```console $ tokentriage validate --config t7-fix-a-streaming-off.yaml warning: routing.rules is empty in route mode; only routing.default_tier and overrides.tier_header will apply mode: route listen: 127.0.0.1:8787 admin: 127.0.0.1:9090 upstreams: 2 - anthropic anthropic https://api.anthropic.com - openai openai https://api.openai.com tiers: 2 (cheapest -> most capable) 0. cheap -> anthropic/claude-haiku-4-5 1. frontier -> openai/gpt-4o rules: 0 OK: configuration is valid ``` Exit status 0. That `warning:` line is also printed twice by the binary, by a different route from the doubled error, and the repeat is elided here too. Remedy B prints the same summary; the two captures differ only in the filename echoed after `--config`. The upstream names, ports and tiers belong to the capture config, so yours will read differently. The follow-up that would let you have both is already named: guardrail framing that understands more than one codec widens the supported set, and this check narrows or folds when that lands. ## What this does not do - **Both flags are off out of the box.** `routing.allow_cross_dialect_streaming` requires `routing.allow_cross_dialect`, and neither defaults to true. With streaming translation off, a streaming request aimed across a dialect boundary goes to the same-dialect default upstream and writes no disclosure block at all. - **A translated stream is not byte-faithful.** That is what translating means. Same-dialect streaming stays byte-identical *through the translation layer* even with translation enabled, and `TestSameDialectStreamUntouchedWithTranslationEnabled` is the test you can run for it. One other layer does touch a same-dialect stream: on a streamed chat request to an `openai`-dialect upstream the gateway splices `stream_options.include_usage` into the outbound body and removes the resulting terminal usage frame from your copy if you did not ask for it, which is how the row carries provider-reported usage instead of an estimate. [When a stream misbehaves](/docs/build/providers-troubleshooting#when-a-stream-misbehaves) has the switch and what turning it off costs. - **No upstream frame is relayed to the client**, on either outcome; `TestTranslatedStreamNeverRelaysUpstreamFrames` pins that. - **There is no retry.** Headers went out with the first frame, so the exchange is terminal by the time a later frame can go wrong. - **A non-2xx upstream reply is a different case.** It relays untranslated, in the served dialect's error shape, with the status preserved. What this page describes is a 200 that went bad after the headers were committed. - **An error event authored by the upstream is terminal too**, and gets no second gateway-authored envelope stacked on top of it. The row then prefers the upstream's own reason, because that is the root cause and the codec failure is its consequence. If `outcome.error` reads like something the provider said, the provider said it. - **This page covers chat streams.** Which pairs can stream at all, and which two dialects TokenTriage drives outward without accepting inward, belong to the matrix.