Skip to content

fix(ai-proxy): prevent status change after 200 on streaming body_reader failure#13655

Open
okaybase wants to merge 2 commits into
apache:masterfrom
okaybase:fix-ai-proxy
Open

fix(ai-proxy): prevent status change after 200 on streaming body_reader failure#13655
okaybase wants to merge 2 commits into
apache:masterfrom
okaybase:fix-ai-proxy

Conversation

@okaybase

@okaybase okaybase commented Jul 3, 2026

Copy link
Copy Markdown
Member

Description

When an AI provider's SSE stream stalls mid-response (e.g. the upstream sends a
few chunks then hangs), the body_reader() call in parse_streaming_response
times out. The error handler at ai-providers/base.lua:551 unconditionally
returns a transport-layer error code via transport_http.handle_error(err)
(504 for timeout, 500 for other errors).

However, before entering parse_streaming_response, the caller at
ai-proxy/base.lua:350 has already set the Content-Type response header.
If at least one SSE chunk has been dispatched to the downstream client, the
HTTP response headers — including status 200 — have already been flushed.
Returning a 504/500 after this point causes nginx to emit:
attempt to set status 504 via ngx.exit after sending out the response status 200
attempt to set ngx.status after sending out response headers

Checklist

  • I have explained the need for this PR and the problem it solves
  • I have explained the changes or the new features added to this PR
  • I have added tests corresponding to this change
  • I have verified that this change is backward compatible (If not, please discuss on the APISIX mailing list first)

@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. bug Something isn't working labels Jul 3, 2026
@okaybase okaybase changed the title fix(ai-proxy): prevent status-after-200 error on body_reader timeout in streaming response fix(ai-proxy): prevent status change after 200 on streaming body_reader failure Jul 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant