Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add thread safety section to flaky test docs #12359

Merged
merged 5 commits into from
May 24, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -289,6 +289,7 @@ Mike Lundy
Milan Lesnek
Miro Hrončok
mrbean-bremen
Nathan Goldbaum
Nathaniel Compton
Nathaniel Waisbrot
Ned Batchelder
Expand Down
2 changes: 2 additions & 0 deletions changelog/12356.doc.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Added a subsection to the documentation for debugging flaky tests to mention
lack of thread safety in pytest as a possible source of flakyness.
6 changes: 6 additions & 0 deletions doc/en/explanation/flaky.rst
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,12 @@ Overly strict assertion

Overly strict assertions can cause problems with floating point comparison as well as timing issues. :func:`pytest.approx` is useful here.

Pytest is not thread safe
~~~~~~~~~~~~~~~~~~~~~~~~~

Pytest is not designed to be safe to use in a multithreaded environment. Multiple pytest tests cannot run simultaneously in different threads within a single Python process and pytest assumes that only one test per process is ever executing.
nicoddemus marked this conversation as resolved.
Show resolved Hide resolved

It is possible to use threads within a single test, but care must be taken to avoid using primitives provided by pytest inside a multithreaded context. For example, :func:`pytest.warns` is not thread safe because it is implemented using the standard library :class:`warnings.catch_warnings` context manager, which is not thread safe. Fixtures are also not automatically thread safe and care should be taken sharing the values returned by fixtures between threads. If you are running a test that uses threads and are seeing flaky test results, do not discount the possibility that the test is implicitly using global state in pytest itself.
nicoddemus marked this conversation as resolved.
Show resolved Hide resolved

Pytest features
^^^^^^^^^^^^^^^
Expand Down
Loading