fix(tests): pin test_runtime WAL to tmp_path to avoid cross-Python flake#62
Closed
maltsev-dev wants to merge 1 commit into
Closed
fix(tests): pin test_runtime WAL to tmp_path to avoid cross-Python flake#62maltsev-dev wants to merge 1 commit into
maltsev-dev wants to merge 1 commit into
Conversation
The test_runtime fixture in test_protect_branches.py built a real NullRunRuntime(api_key, _test_mode=True) without going through the mock_api conftest. The runtime's Transport.start() calls _replay_from_wal() which reads /tmp/nullrun.wal (the default NULLRUN_WAL_PATH fallback). If a previous test run in a different Python version (3.10 or 3.12) had persisted a non-empty WAL, the 3.11 worker would replay those events to a real HTTP endpoint, get HTTP 401, and the fixture would fail at setup with NullRunAuthError: nullrun.breaker.exceptions.NullRunAuthError: Invalid API key This bit CI on 2026-07-11 (run 29156199607): tests 3.10 + 3.12 passed, test 3.11 failed with that error in the fixture setup of test_enforce_sensitive_tool_dict_with_fallback_fail_open. The 3.10/3.12 runs cleared the global /tmp/nullrun.wal by reading it first, so 3.11 picked up the next writer. Order- dependent; flaky on the matrix. Pin NULLRUN_WAL_PATH to a tmp_path-scoped file so each test session reads its own fresh empty WAL. Resolves the flake without touching SDK source (no production code change). Verified locally: - pytest tests/test_protect_branches.py -> 43/43 pass - with pre-seeded stale /tmp/nullrun.wal, the previously failing test now passes. No public API change. No SDK_MIN_VERSION bump. Backends on 1.0.0 keep working unchanged. Recommended: 0.13.6 (no version bump needed for a test-only fix).
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See https://github.com/nullrunio/nullrun-sdk-python/actions/runs/29156199607 for the original failure (3.11 only).
Cherry-pick of
41a16f7fromrelease/0.13.6. Same fix, same rationale. No public API change.