2.10.70: pytest fails because it cannot find tests

[kloczek]

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

• python3 -sBm build -w --no-isolation
• because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
• install .whl file in </install/prefix> using installer module
• run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
• build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-construct-2.10.70-4.fc36.x86_64/usr/lib64/python3.9/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-construct-2.10.70-4.fc36.x86_64/usr/lib/python3.9/site-packages
+ /usr/bin/pytest -ra -m 'not network' --import-mode=importlib
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.9.18, pytest-8.1.1, pluggy-1.4.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/construct-2.10.70
plugins: benchmark-4.0.0
collected 0 items / 15 errors

========================================================================================== ERRORS ===========================================================================================
_________________________________________________________________ ERROR collecting tests/deprecated_gallery/test_formats.py _________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/deprecated_gallery/test_formats.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/deprecated_gallery/test_formats.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
________________________________________________________________ ERROR collecting tests/deprecated_gallery/test_protocols.py ________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/deprecated_gallery/test_protocols.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/deprecated_gallery/test_protocols.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
______________________________________________________________________ ERROR collecting tests/gallery/test_gallery.py _______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/gallery/test_gallery.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/gallery/test_gallery.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
_________________________________________________________________________ ERROR collecting tests/lib/test_binary.py _________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/lib/test_binary.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/lib/test_binary.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
_______________________________________________________________________ ERROR collecting tests/lib/test_bitstream.py ________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/lib/test_bitstream.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/lib/test_bitstream.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
____________________________________________________________________ ERROR collecting tests/lib/test_containers_dict.py _____________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/lib/test_containers_dict.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/lib/test_containers_dict.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
____________________________________________________________________ ERROR collecting tests/lib/test_containers_list.py _____________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/lib/test_containers_list.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/lib/test_containers_list.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
__________________________________________________________________________ ERROR collecting tests/lib/test_hex.py ___________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/lib/test_hex.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/lib/test_hex.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
_______________________________________________________________________ ERROR collecting tests/lib/test_py3compat.py ________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/lib/test_py3compat.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/lib/test_py3compat.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
_________________________________________________________________________ ERROR collecting tests/lib/test_search.py _________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/lib/test_search.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/lib/test_search.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
_________________________________________________________________________ ERROR collecting tests/test_benchmarks.py _________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/test_benchmarks.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_benchmarks.py:3: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
__________________________________________________________________________ ERROR collecting tests/test_compiler.py __________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/test_compiler.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_compiler.py:3: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
____________________________________________________________________________ ERROR collecting tests/test_core.py ____________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/test_core.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_core.py:3: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
____________________________________________________________________________ ERROR collecting tests/test_expr.py ____________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/test_expr.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_expr.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
______________________________________________________________________ ERROR collecting tests/test_multiprocessing.py _______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/construct-2.10.70/tests/test_multiprocessing.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
tests/test_multiprocessing.py:1: in <module>
    from tests.declarativeunittest import *
E   ModuleNotFoundError: No module named 'tests'
================================================================================== short test summary info ==================================================================================
ERROR tests/deprecated_gallery/test_formats.py
ERROR tests/deprecated_gallery/test_protocols.py
ERROR tests/gallery/test_gallery.py
ERROR tests/lib/test_binary.py
ERROR tests/lib/test_bitstream.py
ERROR tests/lib/test_containers_dict.py
ERROR tests/lib/test_containers_list.py
ERROR tests/lib/test_hex.py
ERROR tests/lib/test_py3compat.py
ERROR tests/lib/test_search.py
ERROR tests/test_benchmarks.py
ERROR tests/test_compiler.py
ERROR tests/test_core.py
ERROR tests/test_expr.py
ERROR tests/test_multiprocessing.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 15 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
==================================================================================== 15 errors in 0.50s =====================================================================================

List of installed modules in build env:

Package                       Version
----------------------------- -----------
alabaster                     0.7.16
arrow                         1.3.0
Babel                         2.14.0
build                         1.1.1
cffi                          1.16.0
charset-normalizer            3.3.2
cloudpickle                   3.0.0
cryptography                  42.0.5
distro                        1.9.0
dnf                           4.19.0
docutils                      0.20.1
exceptiongroup                1.1.3
gpg                           1.23.2
idna                          3.6
imagesize                     1.4.1
importlib_metadata            7.0.1
iniconfig                     2.0.0
installer                     0.7.0
Jinja2                        3.1.3
libdnf                        0.73.0
lz4                           4.3.3
MarkupSafe                    2.1.3
numpy                         1.26.5
packaging                     24.0
pluggy                        1.4.0
ply                           3.11
py-cpuinfo                    9.0.0
pycparser                     2.21
Pygments                      2.17.2
pyproject_hooks               1.0.0
pytest                        8.1.1
pytest-benchmark              4.0.0
python-dateutil               2.9.0.post0
requests                      2.31.0
ruamel.yaml                   0.18.5
ruamel.yaml.clib              0.2.8
setuptools                    69.1.1
snowballstemmer               2.2.0
Sphinx                        7.2.6
sphinxcontrib-applehelp       1.0.8
sphinxcontrib-devhelp         1.0.5
sphinxcontrib-htmlhelp        2.0.5
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.7
sphinxcontrib-serializinghtml 1.1.10
tokenize_rt                   5.2.0
tomli                         2.0.1
urllib3                       1.26.18
wheel                         0.43.0
zipp                          3.17.0

Please let me know if you need more details or want me to perform some diagnostics.

[franzhaas]

I think this might be related to this.:
#1068

would you like to try with pre version 8 pytest?

[kloczek]

would you like to try with pre version 8 pytest?

Sorry but I'm only interested latest pytest 8.x.

[franzhaas]

The problem is that due to the changes of pytest on how tests get discovered in version 8 constructs tests don't work anymore.

i belive this is the best solution with little changes short term.:

export PYTHONPATH=.
py.test tests --benchmark-disable --showlocals --verbose

works for me with pytest 8.1.1

[kloczek]

export PYTHONPATH=.
py.test tests --benchmark-disable --showlocals --verbose

works for me with pytest 8.1.1

Issue only is that other modules does not need to altering $PYTHONPATH.

Just build updated pycparser (https://github.com/eliben/pycparser/) and it has separated test suite files in tests/ and it works without PYTHONPATH=.

[franzhaas]

There are breaking changes between pytest 7.x.y and 8.u.v and we have 2 ways to deal with those at the moment. (modifying the pythonpath, and using 7.x.y)

If you want to have a deeper insight, or research a more elegant solution, I think you would be better off discussing this with pytest (https://github.com/pytest-dev/pytest).

Surely other projects have not been affected by these breaking changes without a depreciation period, and others might have already adapted to them...

Your best options are probably to either live with one of the solutions suggested, or make a PR with a solution you like and lobby for it.

[kloczek]

There are breaking changes between pytest 7.x.y and 8.u.v and we have 2 ways to deal with those at the moment. (modifying the pythonpath, and using 7.x.y)

I understand .. however you can find plenty of examples with modules test suites which are OK with pytest 7.x and 8.x.
I have already packaged +1.25k python modules as rpm packages. In that sample only about 5-7 are with pytest tests modules location during pytest execution.

[franzhaas]

https://github.com/franzhaas/construct/tree/speedOptimisations

the commit f4a7867
shows another solution for this problem...