fix: handle mid-stream error events in OpenAI SSE streaming#8031
Merged
fix: handle mid-stream error events in OpenAI SSE streaming#8031
Conversation
When an OpenAI-compatible server sends an error object mid-stream
(e.g. {"error": {"message": "..."}}) the deserializer would fail
trying to parse it as a StreamingChunk, crashing with "Stream decode
error". This format is used by vLLM, SGLang, Exo, and OpenAI itself.
Check for an error key before attempting StreamingChunk deserialization,
matching the pattern used by the official OpenAI Python client.
Closes #8021
Signed-off-by: Clyde <spitfire55@users.noreply.github.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Parse JSON to Value once, then use from_value for StreamingChunk
to avoid deserializing the same bytes twice on the happy path
- Detect vLLM's error shape: {"object":"error", "message":"..."}
in addition to OpenAI/SGLang/Exo {"error":{"message":"..."}}
- Add three tests covering OpenAI format, vLLM format, and error-as-
first-chunk via a run_streaming_test_expecting_error helper
Signed-off-by: Douwe Osinga <douwe@squareup.com>
- parse_streaming_chunk returns ProviderError directly, using ServerError for mid-stream error payloads instead of a generic anyhow error - call site in openai_compatible downcasts to ProviderError before fallback-wrapping, so the error prefix is never doubled - drop trivial format comments from parse_streaming_chunk - collapse three separate async tests into one test_case table Signed-off-by: Douwe Osinga <douwe@squareup.com>
alexhancock
approved these changes
Mar 20, 2026
elijahsgh
pushed a commit
to elijahsgh/goose
that referenced
this pull request
Mar 21, 2026
Signed-off-by: Clyde <spitfire55@users.noreply.github.com> Signed-off-by: Douwe Osinga <douwe@squareup.com> Co-authored-by: Clyde <clyde@Clydes-Mac-Studio.local> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Douwe Osinga <douwe@squareup.com> Signed-off-by: esnyder <elijah.snyder1@gmail.com>
elijahsgh
pushed a commit
to elijahsgh/goose
that referenced
this pull request
Mar 21, 2026
Signed-off-by: Clyde <spitfire55@users.noreply.github.com> Signed-off-by: Douwe Osinga <douwe@squareup.com> Co-authored-by: Clyde <clyde@Clydes-Mac-Studio.local> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Douwe Osinga <douwe@squareup.com> Signed-off-by: esnyder <elijah.snyder1@gmail.com>
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Picks up #8027 from @spitfire55 and adds three improvements:
Eliminate double-parse — parse JSON to
Valueonce, then useserde_json::from_valueforStreamingChunkrather than callingfrom_stra second time on the happy path.Handle vLLM error format — in addition to the OpenAI/SGLang/Exo shape
{"error": {"message": "..."}}, also detect vLLM's top-level shape{"object": "error", "message": "..."}.Add tests — three new async tests using a
run_streaming_test_expecting_errorhelper that drivesresponse_to_streaming_messagewith a mid-stream error line and asserts the error message propagates correctly:Closes #8027