Tests fail on Python 3.5 and Python 3.6 due to namespace package confusion

[berdario]

======================================================================
ERROR: test_wo_parents (zope.configuration.tests.test_docutils.Test_makeDocStructures)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/zope.configuration/src/zope/configuration/tests/test_docutils.py", line 61, in test_wo_parents
    from zope.interface import Interface
ModuleNotFoundError: No module named 'zope.interface'

----------------------------------------------------------------------
Ran 252 tests in 0.195s

FAILED (errors=229)
Test failed: <unittest.runner.TextTestResult run=252 errors=229 failures=0>
error: Test failed: <unittest.runner.TextTestResult run=252 errors=229 failures=0>

I also opened an issue here, hoping that they might be able to help

[mgedmin]

Hm, I'm seeing this with both Python 3.5 and 3.6, when I run tox -e py35,py36. Python 2.7 or 3.4 (i.e. tox -e py27,py34) work fine.

[mgedmin]

I cannot reproduce any failures when running

virtualenv -p python3.6 env
env/bin/python bootstrap.py
bin/buildout
bin/test

How are you running the test suite, @berdario?

[mgedmin]

Strangely enough, python3.6 setup.py test also passess all the tests (and same for python3.5). It's just tox that's broken -- and tox just runs python setup.py test in a virtualenv.

I'm stumped.

[mgedmin]

This works:

virtualenv -p python3.6 env
env/bin/python setup.py test

This fails:

virtualenv -p python3.6 env
env/bin/pip install .
env/bin/python setup.py test

[mgedmin]

This works again:

virtualenv -p python3.6 env
env/bin/pip install -e .
env/bin/python setup.py test

[mgedmin]

I added some debug prints to setup.py, and, well, sys.path looks like this:

/home/mg/src/zopefoundation/zope.configuration/src
/home/mg/src/zopefoundation/zope.configuration
/home/mg/src/zopefoundation/zope.configuration/.tox/py36/lib/python36.zip
/home/mg/src/zopefoundation/zope.configuration/.tox/py36/lib/python3.6
/home/mg/src/zopefoundation/zope.configuration/.tox/py36/lib/python3.6/lib-dynload
/home/mg/opt/python36/lib/python3.6
/home/mg/src/zopefoundation/zope.configuration/.tox/py36/lib/python3.6/site-packages

and if you do import zope and look at zope.__path__, you see

['/home/mg/src/zopefoundation/zope.configuration/src/zope']

The source tree has a src/zope/__init__.py with the setuptools namespace package magic.

/home/mg/src/zopefoundation/zope.configuration/.tox/py36/lib/python3.6/site-packages/zope does not have a __init__.py because pip doesn't create one for namespace packages on Python 3.sufficientlyhighnumber because it's not needed after PEP-420 was implemented in Python 3.3.

This, of course, doesn't explain why tox -e py3.4 doesn't fail. On 3.4, import zope produces a zope.__path__ that contains both locations:

['/home/mg/src/zopefoundation/zope.configuration/src/zope', '/home/mg/src/zopefoundation/zope.configuration/.tox/py34/lib/python3.4/site-packages/zope']

[berdario]

I'm just running the test suite with python setup.py test in a brand new virtualenv

For me it works on Python3.5, and fails on Python3.6

I suspect that you might get different results with Tox due to it reusing an old virtualenv (and this one having different versions of setuptools), do things change if you use tox's --recreate option?

[mgedmin]

I did try with tox -re py34,py35,py36 and I saw 3.4 pass but 3.5 and 3.6 fail.

I also did rm -rf env before doing each of the virtualenv -p python3.6 env tests I mentioned earlier.

[warsaw]

I ran into a similar problem with lazr.config and lazr.delegates. Posted about it on the TiP mailing list and commented on #912

[bowlofeggs]

This is also failing in Fedora's buildsystem (which runs the tests by dnf installing the dependencies and then running /usr/bin/python3 setup.py test) in the same manner, ModuleNotFoundError: No module named 'zope.interface'.

[kathychurch]

I don't use zope, but am having same issue with tox and Python 3.5 and 3.6 for our namespace packages. Hoping you guys find a fix or work-around!

[mgedmin]

tox --develop is the workaround that works for us.

We made it default in our tox.ini in 6e09fee.

[kathychurch]

While the --develop option does make the tests pass, unfortunately that won't work for us because we use the zip file from distshare produced from the tox test as a dependency in subsequent tests on other packages.