fix(ai-proxy): return 502 when streaming converter receives mismatched response format#13229
Merged
Merged
Conversation
…d response format When a protocol converter is active (e.g., Anthropic-to-OpenAI) and the upstream returns SSE events in an unexpected format that the target protocol parser cannot parse, the gateway would crash with a 500 error in the balancer phase because no output was ever sent to the client. This happens because: 1. The target protocol's SSE parser skips all unrecognized events 2. The converter never receives any parsed events to convert 3. No ngx.print() is called, so the response is never committed 4. nginx falls through to proxy_pass/balancer with nil upstream_conf Fix: Track whether any output was produced during converter-mode streaming. If the stream ends without any output, return 502 (Bad Gateway) with a descriptive error message instead of letting nginx fall through to the balancer.
dosubot
Bot
added
size:L
There was a problem hiding this comment.
Pull request overview
This PR hardens ai-proxy streaming behavior in protocol-converter mode so that when the upstream SSE stream is in an unexpected format (and therefore produces no convertible output), APISIX returns 502 instead of leaving the response uncommitted and falling through into nginx’s balancer phase (leading to a crash/500).
Changes:
- Track whether any converted streaming output was emitted; if the stream ends with no output in converter mode, return 502 and log an error.
- Handle EOF buffered SSE remainder by converting and emitting any remaining events in converter mode.
- Add a regression test covering the mismatched upstream SSE format scenario.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
apisix/plugins/ai-providers/base.lua |
Adds “no converted output” detection in converter-mode streaming and returns 502 to prevent nginx balancer fallthrough/crash. |
t/plugin/ai-proxy-protocol-conversion.t |
Adds a streaming regression test for mismatched upstream SSE format when conversion is active. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Return descriptive error message as second return value from both 502 paths - Fix test: remove ngx.status propagation so outer response stays 200
The EOF remainder only contains incomplete SSE data that won't decode to valid events, so converter processing there is dead code. Simplified to just check output_sent after the existing usage extraction loop. Also remove unused variable in test.
The previous EOF remainder code tried to sse.decode() an incomplete SSE buffer, which can never produce valid events. Replace with a simple warning log.
nic-6443
requested review from
AlinsRan,
Baoyuantop,
membphis and
shreemaan-abhishek
membphis
approved these changes
Apr 16, 2026
shreemaan-abhishek
approved these changes
Apr 16, 2026
AlinsRan
approved these changes
Apr 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When a protocol converter is active (e.g., Anthropic-to-OpenAI) and the upstream returns SSE events in an unexpected format that the target protocol parser cannot parse, the gateway crashes with a 500 error in the balancer phase.
Root cause: if the target protocol SSE parser skips all upstream events (unrecognized format), no output is ever sent to the client via
ngx.print(). The response never gets committed, so nginx falls through toproxy_pass→ balancer phase, which crashes on nilupstream_conf.Fix: track whether any output was produced during converter-mode streaming in
parse_streaming_response. If the entire stream is consumed without producing any output, return 502 (Bad Gateway) with a descriptive error instead of letting nginx fall through to the balancer.This is scoped to converter mode only — passthrough streaming is not affected.