pytest-asyncio makes pytest collect modules not matching python_files, causing ModuleNotFoundError

[mgorny]

I've noticed this while running bcrypt's test suite. Normally, pytest only collects files matching python_files option, i.e. test_*.py. However, if pytest-asyncio is installed, it seems to attempt to collect all *.py files, effectively causing ModuleNotFoundErrors if some modules aren't found, e.g.:

$ python -m pytest
========================================================= test session starts =========================================================
platform linux -- Python 3.11.7, pytest-7.4.4, pluggy-1.3.0
rootdir: /tmp/bcrypt
plugins: asyncio-0.23.3
asyncio: mode=Mode.STRICT
collected 0 items                                                                                                                     
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/python.py", line 617, in _importtestmodule
INTERNALERROR>     mod = import_path(self.path, mode=importmode, root=self.config.rootpath)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/pathlib.py", line 567, in import_path
INTERNALERROR>     importlib.import_module(module_name)
INTERNALERROR>   File "/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
INTERNALERROR>     return _bootstrap._gcd_import(name[level:], package, level)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
INTERNALERROR>   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
INTERNALERROR>   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
INTERNALERROR>   File "/tmp/bcrypt/src/bcrypt/__init__.py", line 13, in <module>
INTERNALERROR>     from ._bcrypt import (
INTERNALERROR> ModuleNotFoundError: No module named 'bcrypt._bcrypt'
INTERNALERROR> 
INTERNALERROR> The above exception was the direct cause of the following exception:
INTERNALERROR> 
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/main.py", line 271, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>                          ^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/main.py", line 324, in _main
INTERNALERROR>     config.hook.pytest_collection(session=session)
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 152, in _multicall
INTERNALERROR>     return outcome.get_result()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_result.py", line 114, in get_result
INTERNALERROR>     raise exc.with_traceback(exc.__traceback__)
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/main.py", line 335, in pytest_collection
INTERNALERROR>     session.perform_collect()
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/main.py", line 675, in perform_collect
INTERNALERROR>     self.items.extend(self.genitems(node))
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/main.py", line 842, in genitems
INTERNALERROR>     rep = collect_one_node(node)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/runner.py", line 546, in collect_one_node
INTERNALERROR>     ihook.pytest_collectstart(collector=collector)
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_hooks.py", line 493, in __call__
INTERNALERROR>     return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_manager.py", line 115, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 113, in _multicall
INTERNALERROR>     raise exception.with_traceback(exception.__traceback__)
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pluggy/_callers.py", line 77, in _multicall
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>           ^^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/pytest_asyncio/plugin.py", line 626, in pytest_collectstart
INTERNALERROR>     pyobject = collector.obj
INTERNALERROR>                ^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/python.py", line 310, in obj
INTERNALERROR>     self._obj = obj = self._getobj()
INTERNALERROR>                       ^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/python.py", line 528, in _getobj
INTERNALERROR>     return self._importtestmodule()
INTERNALERROR>            ^^^^^^^^^^^^^^^^^^^^^^^^
INTERNALERROR>   File "/tmp/bcrypt/.venv/lib/python3.11/site-packages/_pytest/python.py", line 642, in _importtestmodule
INTERNALERROR>     raise self.CollectError(
INTERNALERROR> _pytest.nodes.Collector.CollectError: ImportError while importing test module '/tmp/bcrypt/src/bcrypt/__init__.py'.
INTERNALERROR> Hint: make sure your test modules/packages have valid Python names.
INTERNALERROR> Traceback:
INTERNALERROR> /usr/lib/python3.11/importlib/__init__.py:126: in import_module
INTERNALERROR>     return _bootstrap._gcd_import(name[level:], package, level)
INTERNALERROR> src/bcrypt/__init__.py:13: in <module>
INTERNALERROR>     from ._bcrypt import (
INTERNALERROR> E   ModuleNotFoundError: No module named 'bcrypt._bcrypt'

======================================================== no tests ran in 0.04s ========================================================

To reproduce:

git clone https://github.com/pyca/bcrypt/
cd bcrypt
python3.11 -m venv .venv
. .venv/bin/activate
pip install '.[tests]'
python -m pytest  # test suite passes
pip install pytest-asyncio
python -m pytest  # crashes

[bnavigator]

Sounds like another manifestation of #729

[seifertm]

Thanks for reporting this. More and more issues are coming up related to the changed import behaviour cause by pytest-asyncio v0.23.

I'll close this as a duplicate of #729 .

[mgorny]

Sure, I'll subscribe to that bug. Thanks.