Broken pytest module

[aperloff]

The pytest module in python 3 seems to be broken. This was noticed in CMSSW_12_1_0_pre3. The commands in [1] are both returning the error message in [2]. A recipe to reproduce the issue is in [3].

[1]

python3 -m pytest --version
python3 -m pytest <pytest file>

[2]

Traceback (most recent call last):
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/python3/3.9.6/lib/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/python3/3.9.6/lib/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/pytest/__main__.py", line 5, in <module>
    raise SystemExit(pytest.console_main())
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/config/__init__.py", line 185, in console_main
    code = main()
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/config/__init__.py", line 143, in main
    config = _prepareconfig(args, plugins)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/config/__init__.py", line 318, in _prepareconfig
    config = pluginmanager.hook.pytest_cmdline_parse(
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pluggy/1.0.0-7276fcfe7a0d40476d4b09e0839e3ad5/lib/python3.9/site-packages/pluggy/_hooks.py", line 265, in __call__
    return self._hookexec(self.name, self.get_hookimpls(), kwargs, firstresult)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pluggy/1.0.0-7276fcfe7a0d40476d4b09e0839e3ad5/lib/python3.9/site-packages/pluggy/_manager.py", line 80, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pluggy/1.0.0-7276fcfe7a0d40476d4b09e0839e3ad5/lib/python3.9/site-packages/pluggy/_callers.py", line 55, in _multicall
    gen.send(outcome)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/helpconfig.py", line 100, in pytest_cmdline_parse
    config: Config = outcome.get_result()
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pluggy/1.0.0-7276fcfe7a0d40476d4b09e0839e3ad5/lib/python3.9/site-packages/pluggy/_result.py", line 60, in get_result
    raise ex[1].with_traceback(ex[2])
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pluggy/1.0.0-7276fcfe7a0d40476d4b09e0839e3ad5/lib/python3.9/site-packages/pluggy/_callers.py", line 39, in _multicall
    res = hook_impl.function(*args)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1003, in pytest_cmdline_parse
    self.parse(args)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1283, in parse
    self._preparse(args, addopts=addopts)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/config/__init__.py", line 1172, in _preparse
    self.pluginmanager.load_setuptools_entrypoints("pytest11")
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pluggy/1.0.0-7276fcfe7a0d40476d4b09e0839e3ad5/lib/python3.9/site-packages/pluggy/_manager.py", line 287, in load_setuptools_entrypoints
    plugin = ep.load()
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/python3/3.9.6/lib/python3.9/importlib/metadata.py", line 77, in load
    module = import_module(match.group('module'))
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/python3/3.9.6/lib/python3.9/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 972, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 228, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1030, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
  File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-anyio/3.3.0-bc1b2435123cb64e06b80339c9a6cb22/lib/python3.9/site-packages/anyio/__init__.py", line 78, in <module>
    from ._core._eventloop import (
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-pytest/6.2.5-c3dd9306238c9e554d8ef71ca6de0160/lib/python3.9/site-packages/_pytest/assertion/rewrite.py", line 170, in exec_module
    exec(co, module.__dict__)
  File "/cvmfs/cms.cern.ch/slc7_amd64_gcc900/external/py3-anyio/3.3.0-bc1b2435123cb64e06b80339c9a6cb22/lib/python3.9/site-packages/anyio/_core/_eventloop.py", line 8, in <module>
    import sniffio
ModuleNotFoundError: No module named 'sniffio'

[3]

cmsrel CMSSW_12_1_0_pre3
cd CMSSW_12_1_0_pre3/src
cmsenv
python3 -m pytest --version

[cmsbuild]

A new Issue was created by @aperloff Alexx Perloff.

@Dr15Jones, @perrotta, @dpiparo, @makortel, @smuzaffar, @qliphy can you please review it and eventually sign/assign? Thanks.

cms-bot commands are listed here

[davidlange6]

It looks like anyio package has dependencies that pip doesn't discover for us. (and the unit test that does import purest doesn't catch). @smuzaffar - perhaps that is because we do not check for install_requires ?
https://github.com/agronholm/anyio/blob/321c6e22e248d1617106d149c1415e54f9f8de40/setup.cfg#L30

[smuzaffar]

@davidlange6 problem is that anyio package has not set any version so when we run pip3 check then it returns

anyio 0.0.0 requires idna, which is not installed.
anyio 0.0.0 requires sniffio, which is not installed.

while https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_12_1_X/master/build-with-pip.file#L107 we look for ^${PKG_NAME} %{realversion . This check did not find any requirements and that is why cmsBuild did not complain. I will patch anyio sources to set the correct version

[davidlange6]

Hum, google suggests this might be due to our setuptools being old.. (something I can get back to next week)

…

On Oct 6, 2021, at 6:13 PM, Malik Shahzad Muzaffar ***@***.***> wrote: @davidlange6 problem is that anyio package has not set any version so when we run pip3 check then it returns anyio 0.0.0 requires idna, which is not installed. anyio 0.0.0 requires sniffio, which is not installed. while https://github.com/cms-sw/cmsdist/blob/IB/CMSSW_12_1_X/master/build-with-pip.file#L107 we look for ^${PKG_NAME} %{realversion . This check did not find any requirements and that is why cmsBuild did not complain. I will patch anyio sources to set the correct version — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or unsubscribe. Triage notifications on the go with GitHub Mobile for iOS or Android.

[smuzaffar]

#7366 should fix this issue