fix(ci): exclude performance tests from release workflow#184
Merged
Conversation
The release workflow ran `pytest -m "not real_api"`, which still included `test_performance.py` (marked module-level `pytestmark = pytest.mark.performance`). These tests assert wall-clock thresholds — most notably `test_config_load_time` at <100ms — that routinely flake on shared GitHub Actions runners (observed at 0.127s and 0.145s on consecutive runs of the v0.1.15 release). `ci.yml` already excludes them via `-m "not real_api and not performance"` with an inline comment explaining the rationale. Bring `release.yml` into parity so tagged releases are not gated on flaky timing tests. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The release workflow ran
pytest -m "not real_api", which still picked uptests/test_performance.py(entire module is markedpytestmark = pytest.mark.performance). These tests assert wall-clock thresholds — most notablytest_config_load_timeat <100ms — that routinely flake on shared GitHub Actions runners. Observed on the v0.1.15 release run 25826854283:ci.yml:115already excludes them via-m "not real_api and not performance"with an inline comment spelling out exactly this concern. This PR bringsrelease.ymlinto parity.Diff
Follow-up
After merge: delete the
v0.1.15tag and re-tag the new main HEAD so the release workflow re-runs against the fixed config.