fix(streamable_http): expose session_idle_timeout and don't reap active sessions#2510
Open
BuildWithAbid wants to merge 1 commit intomodelcontextprotocol:mainfrom
Open
Conversation
…ve sessions Closes modelcontextprotocol#2455. Two linked problems with the existing `session_idle_timeout` feature: 1. The kwarg lived on `StreamableHTTPSessionManager` but was not threaded through `Server.streamable_http_app()` or `MCPServer.streamable_http_app()`, so the canonical high-level API could not configure it. 2. The idle deadline was reset only on incoming requests. A handler whose work outlived the timeout could be cancelled mid-flight, leaving the client waiting for a response that would never arrive. This change: - Adds `session_idle_timeout` to both `streamable_http_app()` wrappers and forwards it to the underlying `StreamableHTTPSessionManager`. - Replaces the per-request "push the deadline forward" logic with a per- transport in-flight counter. The new `_suspend_idle_timeout()` async context manager bumps the counter on entry — setting `idle_scope.deadline = math.inf` on the first concurrent request — and on exit restores `now + session_idle_timeout` once the last in-flight request completes. Both the new-session and existing-session code paths now wrap `transport.handle_request()` with `_suspend_idle_timeout`, so the deadline is suspended for the lifetime of every request, not just bumped at its start. Tests in `tests/server/test_streamable_http_manager.py`: - `test_session_idle_timeout_passthrough_lowlevel` / `_mcpserver` — verify the kwarg propagates through both wrappers. - `test_suspend_idle_timeout_sets_deadline_inf_then_restores` — helper sets `inf` on enter, finite deadline on exit. - `test_suspend_idle_timeout_only_restores_after_last_concurrent_request` — nested suspensions hold the deadline at `inf` until the outermost exit. - `test_suspend_idle_timeout_no_op_without_timeout` — helper is inert when no timeout is configured. - `test_long_running_request_outlives_idle_timeout` — end-to-end via `httpx.ASGITransport` + `Client`: a tool whose handler sleeps 4× the timeout still completes successfully (regression for problem 2). - `test_idle_session_reaped_after_request_completes` — the session is still reaped once the request finishes (existing behavior preserved).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #2455
Fixes both problems described in the issue:
Problem 1:
session_idle_timeoutnot exposed bystreamable_http_app()StreamableHTTPSessionManager(...)acceptedsession_idle_timeout, but neitherServer.streamable_http_app(...)norMCPServer.streamable_http_app(...)did. So the canonical high-level API could not configure a supported session-manager feature.This PR threads
session_idle_timeout: float | None = Nonethrough both wrappers and forwards it to the underlyingStreamableHTTPSessionManager.Problem 2: active requests cancelled by the idle timeout
The previous logic reset the deadline only at the start of each incoming request. A handler whose work outlived
session_idle_timeoutwould be cancelled mid-flight; the client waited until session termination instead of receiving the tool result.Replaced the deadline-bumping logic with a per-transport in-flight counter and a
_suspend_idle_timeout()async context manager:idle_active_requests += 1. If this is the first concurrent request, setidle_scope.deadline = math.inf.idle_active_requests -= 1. If this was the last in-flight request, restorenow + session_idle_timeout.Both the new-session and existing-session code paths now wrap
transport.handle_request(...)with_suspend_idle_timeout, so the deadline is suspended for the lifetime of every request.Test plan
All in
tests/server/test_streamable_http_manager.py:test_session_idle_timeout_passthrough_lowlevel— verifies the kwarg propagates fromServer.streamable_http_app()to the manager.test_session_idle_timeout_passthrough_mcpserver— same forMCPServer.streamable_http_app().test_suspend_idle_timeout_sets_deadline_inf_then_restores— direct unit test of the helper: deadline isinfduring the request, finite (≈now + timeout) after.test_suspend_idle_timeout_only_restores_after_last_concurrent_request— nested suspensions keep the deadline atinfuntil the outermost exit; counter logic correctness.test_suspend_idle_timeout_no_op_without_timeout— helper is inert when no timeout is configured.test_long_running_request_outlives_idle_timeout— regression for problem 2: end-to-end viahttpx.ASGITransport+Client, a tool whose handler sleeps 4× the timeout still completes successfully.test_idle_session_reaped_after_request_completes— the session is still reaped once the request finishes, so existing reaping behavior is preserved.Local results:
```
tests/server/test_streamable_http_manager.py 18 passed
tests/server/ + tests/shared/test_streamable_http.py 542 passed
ruff check + ruff format clean
pyright 0 errors, 0 warnings
```
Notes
session_idle_timeoutis a new kwarg with the sameNonedefault everywhere.session_idle_timeout > 0, mutually exclusive withstateless) still live onStreamableHTTPSessionManager.__init__and continue to raise correctly when invoked through either wrapper.