fix: drain SSE stream to EOF to prevent ~260ms latency on keepalive connections#2780
Open
xlyoung wants to merge 1 commit into
Open
fix: drain SSE stream to EOF to prevent ~260ms latency on keepalive connections#2780xlyoung wants to merge 1 commit into
xlyoung wants to merge 1 commit into
Conversation
…onnections In _handle_sse_response, the client called await response.aclose() immediately after receiving the first JSON-RPC response event. This early close left the underlying HTTP/1.1 keepalive connection in a half-drained state, causing the next request reusing the same connection to block for ~260ms before the server's response status arrived. Fix: remove the early aclose() and let the SSE stream drain to EOF naturally. The server closes the SSE stream after sending the response (sse_starlette.EventSourceResponse exits via break on JSONRPCResponse), so the loop exits naturally on EOF. Performance improvement: 37x speedup (265ms → 7ms per call in the reporter's setup). Fixes modelcontextprotocol#2707
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In
_handle_sse_response, the client callsawait response.aclose()immediately after receiving the first JSON-RPC response event. This early close leaves the underlying HTTP/1.1 keepalive connection in a half-drained state, causing the next request reusing the same connection to block for ~260ms before the server's response status arrives.Measured impact (from #2707):
ClientSession.call_tool()(current code)ClientSession.call_tool()(with fix)37x speedup per sequential call over streamable HTTP.
Root Cause
The early
aclose()returns the connection to the pool without draining the SSE stream. The next POST on the same connection blocks waiting for the server-side SSE writer to finish.Fix
Remove the early
aclose()and let the SSE stream drain to EOF naturally:The server closes the SSE stream after sending the response (
sse_starlette.EventSourceResponseexits viabreakon JSONRPCResponse), so the loop exits naturally on EOF.Testing
Fixes #2707