Tests fail with ImportPathMismatchError when PIP_USE_FEATURE=2020-resolver

[jaraco]

pytest-checkdocs master $ $PIP_USE_FEATURE='2020-resolver'
pytest-checkdocs master $ tox
python develop-inst-noop: /Users/jaraco/code/main/pytest-checkdocs
python installed: appdirs==1.4.4,attrs==20.3.0,black==20.8b1,click==7.1.2,coverage==5.3,docutils==0.16,filelock==3.0.12,flake8==3.8.4,iniconfig==1.1.1,jaraco.apt==2.0,jaraco.classes==3.1.0,jaraco.collections==3.0.0,jaraco.context==3.0.0,jaraco.functools==3.0.1,jaraco.test==3.2.0,jaraco.text==3.2.0,mccabe==0.6.1,more-itertools==8.6.0,mypy==0.790,mypy-extensions==0.4.3,packaging==20.4,pathspec==0.8.1,pluggy==0.13.1,py==1.9.0,pycodestyle==2.6.0,pyflakes==2.2.0,pyparsing==2.4.7,pytest==6.1.2,pytest-black==0.3.12,-e git+gh://jaraco/pytest-checkdocs@3df4d42ce3628297c061112ed17f45018a8b95d7#egg=pytest_checkdocs,pytest-cov==2.10.1,pytest-flake8==1.0.6,pytest-mypy==0.8.0,pytz==2020.4,regex==2020.11.13,six==1.15.0,tempora==4.0.1,toml==0.10.2,typed-ast==1.4.1,typing-extensions==3.7.4.3,yg.lockfile==2.3,zc.lockfile==2.0
python run-test-pre: PYTHONHASHSEED='2394216951'
python run-test: commands[0] | pytest
======================================================= test session starts ========================================================
platform darwin -- Python 3.9.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
cachedir: .tox/python/.pytest_cache
rootdir: /Users/jaraco/code/main/pytest-checkdocs, configfile: pytest.ini
plugins: mypy-0.8.0, black-0.3.12, flake8-1.0.6, cov-2.10.1, jaraco.test-3.2.0, checkdocs-2.1.2.dev17+g3df4d42
collected 11 items                                                                                                                 

pytest_checkdocs.py ss..                                                                                                     [ 36%]
setup.py .ss.                                                                                                                [ 72%]
docs/conf.py ss.                                                                                                             [100%]

---------- coverage: platform darwin, python 3.9.0-final-0 -----------
Name                  Stmts   Miss  Cover   Missing
---------------------------------------------------
docs/conf.py              3      0   100%
pytest_checkdocs.py      46     27    41%   1-14, 23-24, 29-30, 36-40, 45-46, 51-57, 63, 66, 73-74
---------------------------------------------------
TOTAL                    49     27    45%

=============================================================== mypy ===============================================================

Success: no issues found in 3 source files
=================================================== 5 passed, 6 skipped in 1.25s ===================================================
_____________________________________________________________ summary ______________________________________________________________
  python: commands succeeded
  congratulations :)

pytest-checkdocs master $ tox -r
python recreate: /Users/jaraco/code/main/pytest-checkdocs/.tox/python
python develop-inst: /Users/jaraco/code/main/pytest-checkdocs
python installed: appdirs==1.4.4,attrs==20.3.0,black==20.8b1,click==7.1.2,coverage==5.3,docutils==0.16,filelock==3.0.12,flake8==3.8.4,iniconfig==1.1.1,jaraco.apt==2.0,jaraco.classes==3.1.0,jaraco.collections==3.0.0,jaraco.context==3.0.0,jaraco.functools==3.0.1,jaraco.test==3.2.0,jaraco.text==3.2.0,mccabe==0.6.1,more-itertools==8.6.0,mypy==0.790,mypy-extensions==0.4.3,packaging==20.4,pathspec==0.8.1,pluggy==0.13.1,py==1.9.0,pycodestyle==2.6.0,pyflakes==2.2.0,pyparsing==2.4.7,pytest==6.1.2,pytest-black==0.3.12,pytest-checkdocs==2.1.1,pytest-cov==2.10.1,pytest-flake8==1.0.6,pytest-mypy==0.8.0,pytz==2020.4,regex==2020.11.13,six==1.15.0,tempora==4.0.1,toml==0.10.2,typed-ast==1.4.1,typing-extensions==3.7.4.3,yg.lockfile==2.3,zc.lockfile==2.0
python run-test-pre: PYTHONHASHSEED='3064124592'
python run-test: commands[0] | pytest
======================================================= test session starts ========================================================
platform darwin -- Python 3.9.0, pytest-6.1.2, py-1.9.0, pluggy-0.13.1
cachedir: .tox/python/.pytest_cache
rootdir: /Users/jaraco/code/main/pytest-checkdocs, configfile: pytest.ini
plugins: mypy-0.8.0, black-0.3.12, flake8-1.0.6, checkdocs-2.1.1, cov-2.10.1, jaraco.test-3.2.0
collected 11 items / 1 error / 10 selected                                                                                         

============================================================== ERRORS ==============================================================
_______________________________________________ ERROR collecting pytest_checkdocs.py _______________________________________________
.tox/python/lib/python3.9/site-packages/_pytest/runner.py:310: in from_call
    result = func()  # type: Optional[TResult]
.tox/python/lib/python3.9/site-packages/_pytest/runner.py:340: in <lambda>
    call = CallInfo.from_call(lambda: list(collector.collect()), "collect")
.tox/python/lib/python3.9/site-packages/_pytest/doctest.py:532: in collect
    module = import_path(self.fspath)
.tox/python/lib/python3.9/site-packages/_pytest/pathlib.py:537: in import_path
    raise ImportPathMismatchError(module_name, module_file, path)
E   _pytest.pathlib.ImportPathMismatchError: ('pytest_checkdocs', '/Users/jaraco/code/main/pytest-checkdocs/.tox/python/lib/python3.9/site-packages/pytest_checkdocs.py', PosixPath('/Users/jaraco/code/main/pytest-checkdocs/pytest_checkdocs.py'))

---------- coverage: platform darwin, python 3.9.0-final-0 -----------
Name           Stmts   Miss  Cover   Missing
--------------------------------------------
docs/conf.py       3      0   100%

===================================================== short test summary info ======================================================
ERROR pytest_checkdocs.py - _pytest.pathlib.ImportPathMismatchError: ('pytest_checkdocs', '/Users/jaraco/code/main/pytest-checkdo...
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 1 error during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
========================================================= 1 error in 0.70s =========================================================
ERROR: InvocationError for command /Users/jaraco/code/main/pytest-checkdocs/.tox/python/bin/pytest (exited with code 2)
_____________________________________________________________ summary ______________________________________________________________
ERROR:   python: commands failed

Note that pytest-checkdocs==2.1.1 appears in the installed packages even though pytest-checkdocs in the project under test.

Disabling the 2020 resolver, the expected dependencies appear and tests run fine:

pytest-checkdocs master $ env PIP_USE_FEATURE= tox -r --notest
python recreate: /Users/jaraco/code/main/pytest-checkdocs/.tox/python
python develop-inst: /Users/jaraco/code/main/pytest-checkdocs
python installed: appdirs==1.4.4,attrs==20.3.0,black==20.8b1,click==7.1.2,coverage==5.3,docutils==0.16,filelock==3.0.12,flake8==3.8.4,iniconfig==1.1.1,jaraco.apt==2.0,jaraco.classes==3.1.0,jaraco.collections==3.0.0,jaraco.context==3.0.0,jaraco.functools==3.0.1,jaraco.test==3.2.0,jaraco.text==3.2.0,mccabe==0.6.1,more-itertools==8.6.0,mypy==0.790,mypy-extensions==0.4.3,packaging==20.4,pathspec==0.8.1,pluggy==0.13.1,py==1.9.0,pycodestyle==2.6.0,pyflakes==2.2.0,pyparsing==2.4.7,pytest==6.1.2,pytest-black==0.3.12,-e git+gh://jaraco/pytest-checkdocs@3df4d42ce3628297c061112ed17f45018a8b95d7#egg=pytest_checkdocs,pytest-cov==2.10.1,pytest-flake8==1.0.6,pytest-mypy==0.8.0,pytz==2020.4,regex==2020.11.13,six==1.15.0,tempora==4.0.1,toml==0.10.2,typed-ast==1.4.1,typing-extensions==3.7.4.3,yg.lockfile==2.3,zc.lockfile==2.0
_____________________________________________________________ summary ______________________________________________________________
  python: skipped tests
  congratulations :)

[jaraco]

Removing pytest-checkdocs from the testing extra requirements works around the issue. I suspect there's a regression relating to the resolver when the project declares itself as a dependency.

[jaraco]

I'm able to replicate the issue with pip alone, so I'll report the issue there.