The model broker buffers the entire upstream response before returning: response.read() at services/model-broker/openphone_model_broker.py:311. For stream:true Responses requests (the assistant's realtime path), this means the connection stalls for up to 120s (_proxy_to_openai timeout) and then dumps the buffered body — the assistant cannot stream tokens through broker mode.
Fix
- Detect
Accept: text/event-stream or stream:true in the payload.
- Pass through the upstream response body as SSE chunks (
urllib.response.HTTPResponse.read1 in a loop, flushing per chunk).
- Preserve
Content-Type: text/event-stream, Transfer-Encoding: chunked, and disable buffering in the response wfile.
- Add a smoke test using a synthetic SSE upstream.
Size: M
The model broker buffers the entire upstream response before returning:
response.read()atservices/model-broker/openphone_model_broker.py:311. Forstream:trueResponses requests (the assistant's realtime path), this means the connection stalls for up to 120s (_proxy_to_openaitimeout) and then dumps the buffered body — the assistant cannot stream tokens through broker mode.Fix
Accept: text/event-streamorstream:truein the payload.urllib.response.HTTPResponse.read1in a loop, flushing per chunk).Content-Type: text/event-stream,Transfer-Encoding: chunked, and disable buffering in the response wfile.Size: M