Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Decorate potentially flaky tests with @pytest.mark.flaky from pytest-rerunfailures #2483

Merged
merged 19 commits into from Mar 5, 2024

Conversation

namurphy
Copy link
Member

@namurphy namurphy commented Jan 31, 2024

Description

This PR adds pytest-rerunfailures to the pytest extensions and adds it to some tests of the downloader.

This PR makes it so that test_optical_density_histogram gets rerun when it fails.

Motivation and context

We can use pytest-rerunfailures to repeat flaky tests, including tests that download stuff.

I learned about from episode 213 of the Python Test podcast!

Related issues

This PR was originally related to #1458.

@github-actions github-actions bot added dependencies Related to updating dependency files packaging Related to packaging or distribution labels Jan 31, 2024
@namurphy namurphy changed the title Pytest rerunfailures Use pytest-rerunfailures extension on an intermittent test failure Jan 31, 2024
Copy link

codecov bot commented Jan 31, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 97.36%. Comparing base (20e60aa) to head (c5ad7b0).
Report is 56 commits behind head on main.

❗ Current head c5ad7b0 differs from pull request most recent head 7d5f9f7. Consider uploading reports for the commit 7d5f9f7 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2483      +/-   ##
==========================================
+ Coverage   96.93%   97.36%   +0.42%     
==========================================
  Files         104      104              
  Lines        9163     9361     +198     
==========================================
+ Hits         8882     9114     +232     
+ Misses        281      247      -34     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@github-actions github-actions bot added testing plasmapy.diagnostics Related to the plasmapy.diagnostics subpackage labels Feb 1, 2024
@namurphy
Copy link
Member Author

namurphy commented Feb 1, 2024

pre-commit.ci autofix

@github-actions github-actions bot added the plasmapy.particles Related to the plasmapy.particles subpackage label Feb 1, 2024
@namurphy
Copy link
Member Author

namurphy commented Feb 1, 2024

pre-commit.ci autofix

@namurphy namurphy marked this pull request as ready for review February 1, 2024 00:29
@namurphy namurphy requested a review from a team as a code owner February 1, 2024 00:29
@namurphy namurphy requested review from ejohnson-96 and pheuer and removed request for a team and ejohnson-96 February 1, 2024 00:29
@github-actions github-actions bot added docs PlasmaPy Docs at http://docs.plasmapy.org contributor guide labels Feb 1, 2024
@namurphy namurphy added the status: on hold Issues & PRs that are being intentionally delayed label Feb 1, 2024
@namurphy
Copy link
Member Author

namurphy commented Feb 1, 2024

Putting this on hold for the time being since it may not be needed with #2487.

@namurphy
Copy link
Member Author

namurphy commented Feb 2, 2024

Closing this since #2487 was merged and hopefully fixed the problem in #1458. If it recurs, we can try this again.

@namurphy namurphy closed this Feb 2, 2024
@namurphy namurphy deleted the pytest-rerunfailures branch February 2, 2024 23:49
@namurphy namurphy restored the pytest-rerunfailures branch February 23, 2024 01:01
@namurphy namurphy reopened this Feb 23, 2024
@github-actions github-actions bot added the plasmapy.utils Related to the plasmapy.utils subpackage label Feb 23, 2024
@namurphy namurphy changed the title Use pytest-rerunfailures extension on an intermittent test failure Decorate flaky tests with @pytest.mark.flaky from pytest-rerunfailures Feb 23, 2024
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pheuer — One of the tests in this file failed intermittently in a different PR, so I figured it would be worth using the @pytest.mark.flaky decorator here so that the test will get repeated in case of a failure. It'd probably be worth adding this decorator to all unit tests that require an internet connection.

@namurphy namurphy added status: ready for review PRs that are ready for code review and removed plasmapy.particles Related to the plasmapy.particles subpackage plasmapy.diagnostics Related to the plasmapy.diagnostics subpackage status: on hold Issues & PRs that are being intentionally delayed labels Feb 23, 2024
@namurphy namurphy changed the title Decorate flaky tests with @pytest.mark.flaky from pytest-rerunfailures Decorate potentially flaky tests with @pytest.mark.flaky from pytest-rerunfailures Mar 4, 2024
Copy link
Contributor

@ejohnson-96 ejohnson-96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@namurphy namurphy merged commit cec8444 into PlasmaPy:main Mar 5, 2024
16 checks passed
@namurphy namurphy deleted the pytest-rerunfailures branch March 5, 2024 17:23
@github-actions github-actions bot removed the status: ready for review PRs that are ready for code review label Mar 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contributor guide dependencies Related to updating dependency files docs PlasmaPy Docs at http://docs.plasmapy.org packaging Related to packaging or distribution plasmapy.utils Related to the plasmapy.utils subpackage testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants