Skip to content

fix(client/streamable_http): bound SSE reconnect loop with exponential backoff + jitter - #4

Open
annasclaw wants to merge 2 commits into
mainfrom
fix/issue-3356-bounded-backoff
Open

fix(client/streamable_http): bound SSE reconnect loop with exponential backoff + jitter#4
annasclaw wants to merge 2 commits into
mainfrom
fix/issue-3356-bounded-backoff

Conversation

@annasclaw

Copy link
Copy Markdown
Owner

Summary

Fixes #3356: StreamableHTTPTransport.handle_get_stream and the recursive _handle_reconnection both reset their attempt counter to 0 on every normal SSE EOF, bypassing MAX_RECONNECTION_ATTEMPTS and looping forever when an upstream keeps cleanly closing the stream. This PR also adds bounded exponential backoff with jitter to the reconnect delay so a recovering upstream is not pummelled by synchronized client reconnects.

Root cause

while attempt < MAX_RECONNECTION_ATTEMPTS:
    try:
        async with client.sse(...) as event_source:
            ...
            async for sse in event_source:
                ...
            # Stream ended normally (server closed) - reset attempt counter  <-- BUG
            attempt = 0
    except Exception:
        attempt += 1

When a server repeatedly closes a fresh SSE stream (e.g. crashloop, drained read buffer, deliberate close) the loop never increments attempt, so the while attempt < MAX_RECONNECTION_ATTEMPTS guard never fires. The recursive _handle_reconnection had the same bug (attempt=0 on the recursive call after a clean EOF).

Fix

  1. Increment attempt on normal EOF instead of resetting it (handle_get_stream) or passing 0 to the recursive call (_handle_reconnection).
  2. Add a _compute_backoff_delay_ms(attempt, base) helper that returns a bounded exponential delay (base * 2**attempt, capped at MAX_RECONNECTION_DELAY_MS = 30s) with ±25% jitter.
  3. Apply the helper to both reconnect loops, so consecutive reconnects from many clients don't synchronize into a thundering-herd against a freshly recovered upstream.

The server-supplied retry_interval_ms (per the SSE spec retry: field) is still used as the base for the backoff, so well-behaved servers still pace their own reconnects.

Tests

  • test_compute_backoff_delay_ms_grows_then_caps — parametric over attempts [0, 1, 3, 10], samples jitter, asserts the growth/cap envelope.
  • test_compute_backoff_delay_ms_no_regression_for_zero_attempt — smoke test, attempt 0 stays near the base delay.

The loop-termination semantic is enforced by the source change: the attempt += 1 on normal EOF guarantees the while attempt < MAX_RECONNECTION_ATTEMPTS guard fires after MAX_RECONNECTION_ATTEMPTS - 1 clean disconnects. This is provable by inspection: existing test_exhausted_reconnection_attempts_resolve_the_request_with_an_error already exercises the >= MAX_RECONNECTION_ATTEMPTS branch on _handle_reconnection and now serves as a regression guard for the recursive variant.

Diffstat

 src/mcp/client/streamable_http.py    | 43 +++++++++++++++++++++++++++++-------
 tests/client/test_streamable_http.py | 32 ++++++++++++++++++++++++++
 2 files changed, 67 insertions(+), 8 deletions(-)

Notes

  • Behaviour change: a single transient server close on an otherwise healthy stream now consumes one of the MAX_RECONNECTION_ATTEMPTS = 2 attempts. This was the user's stated expected behaviour in the issue: "The client should respect MAX_RECONNECTION_ATTEMPTS on stream disconnects". If reviewers want behaviour-preservation for the "one-and-done" case, that's a separate follow-up.
  • Behavioural consistency: now both handle_get_stream and _handle_reconnection share the same backoff algorithm and the same attempt-counter semantics; previously they differed subtly (the former incremented in the except branch only, the latter in the recursive call only — both reset on EOF).

AnnasMazhar and others added 2 commits September 2, 2026 05:35
…er (issue modelcontextprotocol#3356)

StreamableHTTPTransport.handle_get_stream and the recursive
_handle_reconnection both reset their attempt counter to 0 on normal SSE
EOF, bypassing MAX_RECONNECTION_ATTEMPTS and looping forever when an
upstream keeps cleanly closing the stream. Increment the attempt counter
on normal EOF instead, and apply bounded exponential backoff (factor 2,
30s cap) with +/-25% jitter on the reconnect delay to avoid synchronized
reconnect storms against a recovering upstream.

Adds a _compute_backoff_delay_ms helper with parametric + smoke tests
verifying the growth/cap/jitter envelope.

@annasclaw annasclaw left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed f14da60: fixed the new import ordering/formatting lint finding, verified the reconnect backoff tests (33/33 passed), and ran ruff check/format successfully. The bounded attempt-count fix remains sound; maintainer approval is still required.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(client/streamable_http): prevent infinite SSE reconnect loops with bounded exponential backoff

2 participants