tests/async_fixtures/test_async_gen_fixtures.py::test_async_gen_fixture_finalized is flaky (in 1.0.0a1)

[mgorny]

While testing 1.0.0a1 on Gentoo, I've noticed that the following test is flaky:

========================================================= test session starts =========================================================
platform linux -- Python 3.11.12, pytest-8.3.5, pluggy-1.5.0 -- /tmp/portage/dev-python/pytest-asyncio-1.0.0_alpha1/work/pytest_asyncio
-1.0.0a1-python3_11/install/usr/bin/python3.11
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase(PosixPath('/tmp/portage/dev-python/pytest-asyncio-1.0.0_alpha1/w
ork/pytest_asyncio-1.0.0a1/.hypothesis/examples'))
rootdir: /tmp/portage/dev-python/pytest-asyncio-1.0.0_alpha1/work/pytest_asyncio-1.0.0a1
configfile: pyproject.toml
testpaths: docs, tests
plugins: xdist-3.6.1
asyncio: mode=Mode.AUTO, asyncio_default_fixture_loop_scope=function, asyncio_default_test_loop_scope=function
created: 12/12 workers
12 workers [166 items]

[…]
============================================================== FAILURES ===============================================================
__________________________________________________ test_async_gen_fixture_finalized ___________________________________________________
[gw0] linux -- Python 3.11.12 /tmp/portage/dev-python/pytest-asyncio-1.0.0_alpha1/work/pytest_asyncio-1.0.0a1-python3_11/install/usr/bi
n/python3.11
tests/async_fixtures/test_async_gen_fixtures.py:37: in test_async_gen_fixture_finalized
    assert mock.called
E   AssertionError: assert False
E    +  where False = <Mock id='139781318067472'>.called
        mock       = <Mock id='139781318067472'>
======================================================= short test summary info =======================================================
FAILED tests/async_fixtures/test_async_gen_fixtures.py::test_async_gen_fixture_finalized - AssertionError: assert False
 +  where False = <Mock id='139781318067472'>.called

It failed for me (on a busy system) roughly 1 out of 3 attempts, once in Python 3.11, twice on 3.13 freethreading.

[mgorny]

After adding --reruns locally, it fails reliably within the same session — that might suggest that it depends on the order in which tests are run.

[seifertm]

Thanks for the report! Historically, the tests weren't isolated against each other. Due to the nature of the tests, they modify global state. This combination turned out to lead to extremely hard-to-debug issues, so we moved to pytest.Testdir and later pytest.Pytester. However, not all of the tests have been ported, yet.

The mentioned test should be wrapped to use Pytester and adjusted if necessary. This needs to happen as part of the v1.0.0 release.

[seifertm]

@mgorny The test was redundant and was removed in v1.0.0.