Skip to content

Conversation

@maxisbey
Copy link
Contributor

Fixes flaky test failures by replacing arbitrary sleep delays with deterministic server readiness polling across all HTTP server tests.

Motivation and Context

Multiple test files were using either fixed time.sleep() delays or manual polling loops with inconsistent implementations to wait for servers to start. This approach was problematic because:

  • The fixed delays were sometimes insufficient in CI environments, causing intermittent failures
  • Fixed delays wasted time when servers started quickly
  • Manual polling implementations were duplicated across test files
  • Tests were non-deterministic and probabilistic rather than reliable

This PR follows the same approach as #1526, which fixed similar issues in test_sse_security.py.

How Has This Been Tested?

  1. Verified the fix: Ran all affected test suites successfully:

    • test_streamable_http_security.py: 7 passed
    • test_sse_security.py: 8 passed
    • test_ws.py: 4 passed
    • test_sse.py: 16 passed, 1 skipped
    • test_streamable_http.py: 33 passed
  2. Consistency check: All test files now use the same centralized wait_for_server() helper

  3. Performance: Tests complete in similar or faster time since they don't wait unnecessarily

Breaking Changes

None. This is purely an internal test improvement with no impact on the public API or user code.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update

Checklist

  • I have read the MCP Documentation
  • My code follows the repository's style guidelines
  • New and existing tests pass locally
  • I have added appropriate error handling
  • I have added or updated documentation as needed

Additional context

The new wait_for_server() helper in tests/test_helpers.py:

  • Actively polls the server port by attempting TCP connections
  • Returns immediately when the server accepts connections (often faster than fixed delays)
  • Uses small 0.01-second intervals between attempts to balance responsiveness and CPU efficiency
  • Has a clear 5-second timeout that raises TimeoutError on true failures
  • Makes tests deterministic instead of probabilistic

This follows the principles outlined in the repository's approach to avoiding flaky tests by using active condition checking rather than arbitrary sleeps.

…tests

Eliminates race conditions in HTTP server tests by replacing fixed sleep delays
with deterministic server readiness polling. This makes tests more reliable and
often faster by eliminating unnecessary waits.

Changes:
- Created shared wait_for_server() helper in tests/test_helpers.py
- Fixed flaky time.sleep(1) in test_streamable_http_security.py
- Replaced duplicate wait_for_server() in test_sse_security.py with shared helper
- Improved consistency by updating all HTTP server test fixtures in:
  - tests/shared/test_ws.py
  - tests/shared/test_sse.py
  - tests/shared/test_streamable_http.py

The wait_for_server() helper:
- Actively polls the server port by attempting TCP connections
- Returns immediately when the server accepts connections
- Uses 0.01-second intervals between attempts
- Has a 5-second timeout that raises TimeoutError on true failures
- Makes tests deterministic instead of probabilistic

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@maxisbey maxisbey enabled auto-merge (squash) October 28, 2025 21:38
Copy link
Member

@johnw188 johnw188 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@maxisbey maxisbey merged commit 3e86edf into main Oct 28, 2025
38 of 39 checks passed
@maxisbey maxisbey deleted the fix-flaky-server-startup-tests branch October 28, 2025 21:42
maxisbey added a commit that referenced this pull request Oct 29, 2025
This change completes the work started in PR #1527 by fixing the
remaining test fixtures that were still using manual socket polling
loops to wait for server startup.

## Changes

Fixed 4 test fixtures that were missed in the previous PR:
- tests/server/fastmcp/test_integration.py: server_transport fixture
- tests/client/test_notification_response.py: non_sdk_server fixture
- tests/client/test_http_unicode.py: running_unicode_server fixture
- tests/shared/test_sse.py: mounted_server and context_server fixtures

All now use the centralized wait_for_server() helper which:
- Actively polls for TCP connection readiness
- Returns immediately when server is ready
- Has clear timeout behavior (5s default)
- Eliminates race conditions from arbitrary sleeps

## Impact

- Removes 49 lines of duplicate manual polling code
- Makes tests more deterministic and reliable
- Reduces wasted time from fixed delays (tests return as soon as ready)
- Follows the patterns documented in FLAKY_TESTS.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants