fix(ai-proxy): prevent status change after 200 on streaming body_reader failure#13655
Open
okaybase wants to merge 2 commits into
Open
fix(ai-proxy): prevent status change after 200 on streaming body_reader failure#13655okaybase wants to merge 2 commits into
okaybase wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
When an AI provider's SSE stream stalls mid-response (e.g. the upstream sends a
few chunks then hangs), the
body_reader()call inparse_streaming_responsetimes out. The error handler at
ai-providers/base.lua:551unconditionallyreturns a transport-layer error code via
transport_http.handle_error(err)(504 for timeout, 500 for other errors).
However, before entering
parse_streaming_response, the caller atai-proxy/base.lua:350has already set theContent-Typeresponse header.If at least one SSE chunk has been dispatched to the downstream client, the
HTTP response headers — including status
200— have already been flushed.Returning a 504/500 after this point causes nginx to emit:
attempt to set status 504 via ngx.exit after sending out the response status 200
attempt to set ngx.status after sending out response headers
Checklist