v0.3.0 - Alpha Release
This release addresses critical test stability issues that were blocking reliable CI/CD execution. What started as intermittent test failures led to a comprehensive refactoring of the test infrastructure, resulting in a more reliable and maintainable codebase.
🔍 The problem: Global state interference in test execution
The test suite exhibited a critical issue: 33 tests would fail when run as part of the full suite, but all tests passed when executed individually. Root cause analysis identified slowapi.Limiter as the culprit—it maintains global state that persists across test executions.
Technical Details:
- Even with unique storage URIs per test,
slowapi.Limiterretains internal state beyond the storage layer - After approximately 18 tests using
testclient, the accumulated request count triggered rate limiting (HTTP 429) in subsequent tests - This created non-deterministic test behavior that made CI/CD unreliable
The issue demonstrated a fundamental problem: tests were not properly isolated, leading to cross-test contamination.
💡 Solution: Three-Layer Isolation Strategy
We implemented a comprehensive three-layer approach to guarantee complete test isolation:
-
Process Isolation: Using
pytest-xdist, each test runs in its own process, eliminating any possibility of shared state between tests. -
Aggressive Monkeypatching: We replace the limiter at the module level to ensure complete isolation, even for libraries with persistent global state.
-
Strategic Organization: We completely reorganized the test structure, clearly separating unit, integration, and end-to-end tests into distinct directories.
🏗️ [New] Test architecture
The test structure was redesigned to follow best practices:
tests/
├── transport/
│ ├── unit/ # Isolated, fast unit tests
│ ├── integration/ # Integration tests with proper isolation
│ └── e2e/ # Complete end-to-end tests
Each test category now has its own namespace, fixtures, and isolation guarantees.
📚 Documentation
Beyond fixing the issue, we documented the entire testing strategy. Created a complete testing guide in docs/testing.md that covers:
- How to organize new tests
- Isolation strategies
- Practical examples for each test type
- Patterns and anti-patterns
Contributors can now quickly understand how to write tests that won't interfere with each other.
🐛 Additional Fixes
During the investigation, we also fixed a critical bug: an UnboundLocalError in server.py related to rate limiter initialization. This type of bug is exactly what flaky tests hide—with stable tests, issues like this are caught immediately.
📊 Results
- 33 failing tests → 0 failing tests
- 610 tests passing consistently
- 100% reliability in full suite execution
- Coverage maintained across all modules (92.47%)
🎉 Impact
The test suite is now stable and deterministic. We can execute all 610 tests with confidence that each test is truly isolated and results are reproducible.
This release doesn't add new features, but significantly improves project stability and reliability. This type of foundational work is essential for long-term project health.
Technical Details
Changed
Test Infrastructure Refactoring (PR #18)
- Test Organization:
- Reorganized test structure with clear separation between unit, integration, and E2E tests
- Created
tests/transport/unit/for isolated unit tests - Created
tests/transport/integration/for integration tests with proper isolation - Created
tests/transport/e2e/for end-to-end tests
- Test Stability:
- Fixed 33 failing tests caused by
slowapi.Limiterglobal state interference - Implemented process isolation using
pytest-xdistto prevent test interference - Added aggressive monkeypatch strategy for complete rate limiter isolation
- Separated rate-limiting tests from core server tests to prevent cross-contamination
- Fixed 33 failing tests caused by
- Documentation:
- Added comprehensive testing guide in
docs/testing.md - Documented test organization strategy and isolation techniques
- Added examples for writing unit, integration, and E2E tests
- Added comprehensive testing guide in
Fixed
- Resolved
UnboundLocalErrorinserver.pyrelated to rate limiter initialization - Fixed test flakiness caused by global state persistence across test runs
- Improved test reliability with proper fixture isolation
Technical Details
- Test Count: 610 tests (all passing)
- Test Execution: Process isolation via
pytest-xdistfor complete state separation - Test Coverage: Maintained comprehensive coverage across all modules (92.47%)
Full Changelog: v0.1.0...v0.3.0