ConftestImportFailure: internal traceback when non-root conftest raises error

[asottile]

an internal traceback is exposed when a nested conftest raises an error (doesn't happen for top level conftest failure)

• a detailed description of the bug or suggestion
• output of pip list from the virtual environment you are using
• pytest and operating system versions
• minimal example if possible

$ pip freeze --all
attrs==19.3.0
importlib-metadata==1.5.0
more-itertools==8.2.0
packaging==20.3
pip==20.0.2
pluggy==0.13.1
py==1.8.1
pyparsing==2.4.6
pytest==5.4.1
setuptools==45.2.0
six==1.14.0
wcwidth==0.1.8
wheel==0.34.2
zipp==3.1.0
$ tree tests
tests
└── wat
    └── conftest.py

1 directory, 1 file
$ cat tests/wat/conftest.py 
import wat

$ pytest tests
============================= test session starts ==============================
platform linux -- Python 3.6.9, pytest-5.4.1, py-1.8.1, pluggy-0.13.1
rootdir: /tmp/y
collected 0 items / 1 error                                                    

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
venv/lib/python3.6/site-packages/_pytest/config/__init__.py:495: in _importconftest
    return self._conftestpath2mod[key]
E   KeyError: PosixPath('/tmp/y/tests/wat/conftest.py')

During handling of the above exception, another exception occurred:
venv/lib/python3.6/site-packages/_pytest/config/__init__.py:501: in _importconftest
    mod = conftestpath.pyimport()
venv/lib/python3.6/site-packages/py/_path/local.py:701: in pyimport
    __import__(modname)
venv/lib/python3.6/site-packages/_pytest/assertion/rewrite.py:152: in exec_module
    exec(co, module.__dict__)
tests/wat/conftest.py:1: in <module>
    import wat
E   ModuleNotFoundError: No module named 'wat'

During handling of the above exception, another exception occurred:
venv/lib/python3.6/site-packages/py/_path/common.py:383: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
venv/lib/python3.6/site-packages/py/_path/common.py:424: in gen
    dirs = self.optsort([p for p in entries
venv/lib/python3.6/site-packages/py/_path/common.py:425: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
venv/lib/python3.6/site-packages/_pytest/nodes.py:506: in _recurse
    ihook = self._gethookproxy(dirpath)
venv/lib/python3.6/site-packages/_pytest/nodes.py:487: in _gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
venv/lib/python3.6/site-packages/_pytest/config/__init__.py:473: in _getconftestmodules
    mod = self._importconftest(conftestpath)
venv/lib/python3.6/site-packages/_pytest/config/__init__.py:509: in _importconftest
    raise ConftestImportFailure(conftestpath, sys.exc_info())
E   _pytest.config.ConftestImportFailure: (local('/tmp/y/tests/wat/conftest.py'), (<class 'ModuleNotFoundError'>, ModuleNotFoundError("No module named 'wat'",), <traceback object at 0x7ff80ffddb08>))
=========================== short test summary info ============================
ERROR  - _pytest.config.ConftestImportFailure: (local('/tmp/y/tests/wat/conft...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.24s ===============================

note that this does not occur with a toplevel conftest (it is caught and reported properly)

$ pytest tests/wat
ImportError while loading conftest '/tmp/y/tests/wat/conftest.py'.
tests/wat/conftest.py:1: in <module>
    import wat
E   ModuleNotFoundError: No module named 'wat'

[SalmonMode]

I might be misunderstanding, but it looks like you're trying to import the package that the conftest is inside of into the conftest, which would be circular.

[asottile]

it has this behaviour for any exception, the concidence in import name was perhaps not the best choice when I wrote up this bug report (I was being too hasty!)

[DahlitzFlorian]

I would say it is related to #7150 and both can be closed due to merging #7244.

[asottile]

seems to still not quite be right, though improved a bit:

$ pytest tests
============================= test session starts ==============================
platform linux -- Python 3.8.2, pytest-5.4.1.dev356+g981b09694, py-1.8.1, pluggy-0.13.1
rootdir: /tmp
collected 0 items / 1 error                                                    

==================================== ERRORS ====================================
________________________ ERROR collecting test session _________________________
venv/lib/python3.8/site-packages/_pytest/config/__init__.py:515: in _importconftest
    mod = conftestpath.pyimport()
venv/lib/python3.8/site-packages/py/_path/local.py:701: in pyimport
    __import__(modname)
venv/lib/python3.8/site-packages/_pytest/assertion/rewrite.py:152: in exec_module
    exec(co, module.__dict__)
tests/x/conftest.py:1: in <module>
    import wat
E   ModuleNotFoundError: No module named 'wat'

The above exception was the direct cause of the following exception:
venv/lib/python3.8/site-packages/py/_path/common.py:383: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
venv/lib/python3.8/site-packages/py/_path/common.py:424: in gen
    dirs = self.optsort([p for p in entries
venv/lib/python3.8/site-packages/py/_path/common.py:425: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
venv/lib/python3.8/site-packages/_pytest/nodes.py:525: in _recurse
    ihook = self._gethookproxy(dirpath)
venv/lib/python3.8/site-packages/_pytest/nodes.py:506: in _gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
venv/lib/python3.8/site-packages/_pytest/config/__init__.py:485: in _getconftestmodules
    mod = self._importconftest(conftestpath)
venv/lib/python3.8/site-packages/_pytest/config/__init__.py:517: in _importconftest
    raise ConftestImportFailure(conftestpath, sys.exc_info()) from e
E   _pytest.config.ConftestImportFailure: ModuleNotFoundError: No module named 'wat' (from /tmp/tests/x/conftest.py)
=========================== short test summary info ============================
ERROR  - _pytest.config.ConftestImportFailure: ModuleNotFoundError: No module...
!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!
=============================== 1 error in 0.25s ===============================

[DahlitzFlorian]

Ah right. I guess it's because we now return the internal traceback if called with --pdb and not every time. I can investigate it further if you wish :)

[DahlitzFlorian]

I identified the _repr_failure_py() function to be the one to be changed. (https://github.com/pytest-dev/pytest/blob/master/src/_pytest/nodes.py#L333)
Added a test for it and adjust an existing test, which expects ConftestImportFailure to be present in the output. PR is coming.