fix(llm): drop extra_body.stream from non-streaming requests (#647) - #657
Merged
lizhengfeng101 merged 1 commit intoAug 1, 2026
Conversation
…#647) Forwarding extra_body.stream to the non-streaming Chat Completions and Anthropic Messages APIs made the server return text/event-stream (SSE) while the SDK's non-streaming New() expects a JSON body, causing every call to fail with "expected destination type of 'string' or '[]byte' for responses with content-type 'text/event-stream;charset=utf-8' that is not 'application/json'". The OpenAIResponsesClient already had this fix (it drops the stream key in CompletionsWithCtx). Apply the same pattern to OpenAIClient and AnthropicClient so the three non-streaming clients behave consistently. For OpenAIClient the streaming decision is preserved: only boolean true triggers NewStreaming (which sets stream=true itself); other value types (string "true", bool false) now have the key dropped from the wire body so the server returns JSON instead of SSE.
Contributor
|
✅ OpenCodeReview: No comments generated. Looks good to me. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When
extra_body.streamis configured, every LLM call fails with:Root cause
The error originates from the SDK's response decoder (both
openai-go/v3andanthropic-sdk-goshare this code via Stainless generation). It surfaces when a non-streaming SDK call (*.New(), which expectsapplication/json) receives an SSE body (text/event-stream).OpenAIClientandAnthropicClientforwarded everyextra_bodykey straight into the wire request body viaWithJSONSet, includingstream. The server therefore answered with SSE, but the client invoked the non-streamingNew()path that expects JSON → decode fails.This was already fixed for
OpenAIResponsesClient— it drops thestreamkey inCompletionsWithCtx(seeresponses_client.go:82-93, with a detailed comment explaining why). This PR applies the same pattern to the other two non-streaming clients so all three behave consistently.Changes
internal/llm/client.go:OpenAIClient.CompletionsWithCtx: skip thestreamkey when mergingextra_bodyinto the request body. The streaming decision is preserved — onlystream: trueas abooltriggers the streaming path (NewStreamingsetsstream=trueitself on the wire); other value types ("true"string,falsebool) now have the key dropped entirely so the server returns JSON instead of SSE.AnthropicClient.CompletionsWithCtx: skip thestreamkey (mirrorsOpenAIResponsesClient— this client has no streaming path at all, so anystreamvalue would break the call).internal/llm/client_test.go:TestOpenAIClient_StreamingRequiresBooleanTrue→TestOpenAIClient_NonStreamingRequestDropsStreamField: now verifiesstreamis absent from the non-streaming request body across three value types (missing /false/"true").TestAnthropicClient_ExtraBodyStreamDropped: verifiesstreamis dropped while otherextra_bodykeys (keep_me,temperature_override) are still forwarded (mirrorsTestOpenAIResponsesClient_ExtraBodyStreamDropped).End-to-end verification
Using the reporter's exact setup (DashScope-compatible Anthropic endpoint,
model: glm-5.2):extra_body.streamocr llm testresult80a5794)true85fccde)trueConnection test successfulType of Change
How Has This Been Tested?
make testpasses locallymodel: glm-5.2andextra_body: {"stream": true})Additional test coverage:
TestOpenAIClient_Streaming*tests (tool call, cancellation, error, usage, reasoning content, etc.) pass unchanged — the streaming path forstream: true(bool) is preserved.internal/llmpackage test suite passes (go test -race -count=1 ./internal/llm/..., ~89s).Checklist
go fmt,go vet)extra_bodydocs inREADME.mdalready accurately describe thatextra_body.stream=trueenables streaming for the Chat Completions client, and that remains unchanged)Related Issues
Relates to #647 — this PR addresses the same error, but the issue is intentionally left open for a short observation period to confirm this fix resolves the underlying problem for all affected users.