pyargs with subpackage breaks fixture (conftest.py?) discovery

[cryvate]

Create a python package named foo with folder structure foo/bar/ with foo/bar containing a conftest.py and test_foo.py where test_foo.py uses a fixture from conftest.py. Now install this package and use --pyargs hci.bar to run the tests. This fails because the fixture is not picked up, and from looking at the logs it seems it is because it thinks conftest.py path includes .../local/... whereas the test path does not include that (those locations inside virtualenvs are symlinked?). I used the following tox.ini to debug:

[testenv]
deps = pytest
whitelist_externals =
    cp
    sed
    rm
ignore_errors = true


# fails because reads conftest from .../local/...
[testenv:fail1]
commands =
    pytest --pyargs foo.bar --debug
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/{envname}/venv/ pytestdebug.log -i
    cp pytestdebug.log pytestdebug-{envname}.log


# succeeds because no .../local/... to get confused (develop)
[testenv:success3]
usedevelop = true
commands =
    pytest --pyargs foo.bar --debug
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/{envname}/venv/ pytestdebug.log -i
    cp pytestdebug.log pytestdebug-{envname}.log

# succeeds because no .../local/... to get confused (removed)
[testenv:success2]
commands =
    rm -rf .tox/{envname}/local
    pytest --pyargs foo.bar --debug
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/{envname}/venv/ pytestdebug.log -i
    cp pytestdebug.log pytestdebug-{envname}.log

# succeeds because it does not have to go up? So path resolution is different?
[testenv:success1]
commands =
    pytest --pyargs foo --debug
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/0x[0-9a-z]\+/0x.../ pytestdebug.log -i
    sed s/{envname}/venv/ pytestdebug.log -i
    cp pytestdebug.log pytestdebug-{envname}.log

This is on pytest 3.2.3, python 2.7.12, Ubuntu 16.04.

Why did I think this might be the issue? I compared the logs (in particular, fail1 to success1) I noticed the difference here of the path in plugin: <module 'foo.bar.conftest' from '<...>/.tox/venv/local/lib/python2.7/site-packages/foo/bar/conftest.py'> and a few others, however my diagnosis might be way off!

I'd be happy to investigate myself if someone has got some pointers of where to start looking.

[cryvate]

I cannot reproduce this on OS X Yosemite Python 2.7.10 nor on Python3 on linux (neither have a local/ in their virtualenvironment) so these symlinks do seem to be the issue.

[cryvate]

In virtualenvs that have a local with symlinks, it appears before the actual lib/ in the path, so imported modules from site-packages will have a __file__ including local.

However, when converting the pyargs a call is made to pkgutil.find_loader which removes the symlink. The solution might be to use importlib instead (which wasn't in 2.6, but the code is minimal so could be vendored until 2.6 support is dropped)

[cryvate]

Using --trace-config makes this much easier to spot.

[RonnyPfannschmidt]

we no longer support 2.6 if you like you can propose a pr for inclusion in the next feature/bugfix release

[cryvate]

I can confirm (I installed the development version of pytest with #3010 merged) fixes this issue, so it can be closed.