New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Decorate potentially flaky tests with @pytest.mark.flaky
from pytest-rerunfailures
#2483
Conversation
pytest-rerunfailures
extension on an intermittent test failure
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2483 +/- ##
==========================================
+ Coverage 96.93% 97.36% +0.42%
==========================================
Files 104 104
Lines 9163 9361 +198
==========================================
+ Hits 8882 9114 +232
+ Misses 281 247 -34 ☔ View full report in Codecov by Sentry. |
pre-commit.ci autofix |
for more information, see https://pre-commit.ci
pre-commit.ci autofix |
Putting this on hold for the time being since it may not be needed with #2487. |
pytest-rerunfailures
extension on an intermittent test failure@pytest.mark.flaky
from pytest-rerunfailures
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@pheuer — One of the tests in this file failed intermittently in a different PR, so I figured it would be worth using the @pytest.mark.flaky
decorator here so that the test will get repeated in case of a failure. It'd probably be worth adding this decorator to all unit tests that require an internet connection.
@pytest.mark.flaky
from pytest-rerunfailures
@pytest.mark.flaky
from pytest-rerunfailures
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
Description
This PR adds
pytest-rerunfailures
to the pytest extensions and adds it to some tests of the downloader.This PR makes it so thattest_optical_density_histogram
gets rerun when it fails.Motivation and context
We can use pytest-rerunfailures to repeat flaky tests, including tests that download stuff.
I learned about from episode 213 of the Python Test podcast!
Related issues
This PR was originally related to #1458.