Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TestNamespaces.test_namespace_package_importable failed with 31.0.0 #884

Closed
felixonmars opened this issue Dec 12, 2016 · 16 comments
Closed

TestNamespaces.test_namespace_package_importable failed with 31.0.0 #884

felixonmars opened this issue Dec 12, 2016 · 16 comments

Comments

@felixonmars
Copy link
Contributor

@felixonmars felixonmars commented Dec 12, 2016

Python 3.5.2
pip 9.0.1
Arch Linux x86_64 latest.

The following new test failure was introduced in 31.0.0:

=================================== FAILURES ===================================
_______________ TestNamespaces.test_namespace_package_importable _______________

self = <setuptools.tests.test_develop.TestNamespaces object at 0x7f9a0d93ab38>
tmpdir = local('/tmp/pytest-of-builduser/pytest-0/test_namespace_package_importa0')

    def test_namespace_package_importable(self, tmpdir):
        """
            Installing two packages sharing the same namespace, one installed
            naturally using pip or `--single-version-externally-managed`
            and the other installed using `develop` should leave the namespace
            in tact and both packages reachable by import.
            """
        pkg_A = namespaces.build_namespace_package(tmpdir, 'myns.pkgA')
        pkg_B = namespaces.build_namespace_package(tmpdir, 'myns.pkgB')
        target = tmpdir / 'packages'
        # use pip to install to the target directory
        install_cmd = [
            'pip',
            'install',
            str(pkg_A),
            '-t', str(target),
        ]
        subprocess.check_call(install_cmd)
        self.install_develop(pkg_B, target)
        namespaces.make_site_dir(target)
        try_import = [
            sys.executable,
            '-c', 'import myns.pkgA; import myns.pkgB',
        ]
        env = dict(PYTHONPATH=str(target))
>       subprocess.check_call(try_import, env=env)

setuptools/tests/test_develop.py:164:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

popenargs = (['/usr/bin/python', '-c', 'import myns.pkgA; import myns.pkgB'],)
kwargs = {'env': {'PYTHONPATH': '/tmp/pytest-of-builduser/pytest-0/test_namespace_package_importa0/packages'}}
retcode = 1
cmd = ['/usr/bin/python', '-c', 'import myns.pkgA; import myns.pkgB']

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.

        The arguments are the same as for the call function.  Example:

        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd)
E           subprocess.CalledProcessError: Command '['/usr/bin/python', '-c', 'import myns.pkgA; import myns.pkgB']' returned non-zero exit status 1

/usr/lib/python3.5/subprocess.py:581: CalledProcessError
----------------------------- Captured stdout call -----------------------------
Processing /tmp/pytest-of-builduser/pytest-0/test_namespace_package_importa0/myns.pkgA
Installing collected packages: myns.pkgA
  Running setup.py install for myns.pkgA: started
    Running setup.py install for myns.pkgA: finished with status 'done'
Successfully installed myns.pkgA-1.0
running develop
running egg_info
creating myns.pkgB.egg-info
writing dependency_links to myns.pkgB.egg-info/dependency_links.txt
writing top-level names to myns.pkgB.egg-info/top_level.txt
writing myns.pkgB.egg-info/PKG-INFO
writing namespace_packages to myns.pkgB.egg-info/namespace_packages.txt
writing manifest file 'myns.pkgB.egg-info/SOURCES.txt'
reading manifest file 'myns.pkgB.egg-info/SOURCES.txt'
writing manifest file 'myns.pkgB.egg-info/SOURCES.txt'
running build_ext
Creating /tmp/pytest-of-builduser/pytest-0/test_namespace_package_importa0/packages/site.py
Creating /tmp/pytest-of-builduser/pytest-0/test_namespace_package_importa0/packages/myns.pkgB.egg-link (link to .)
Adding myns.pkgB 1.0 to easy-install.pth file

Installed /tmp/pytest-of-builduser/pytest-0/test_namespace_package_importa0/myns.pkgB
Processing dependencies for myns.pkgB==1.0
Finished processing dependencies for myns.pkgB==1.0
----------------------------- Captured stderr call -----------------------------
Traceback (most recent call last):
  File "<string>", line 1, in <module>
ImportError: No module named 'myns.pkgB'
========= 1 failed, 206 passed, 12 skipped, 2 xpassed in 15.65 seconds =========
@jaraco
Copy link
Member

@jaraco jaraco commented Dec 12, 2016

Bummer. I'd ask you to help debug, but these tests are particularly difficult to debug. I'll try to find a way to make the test failures more informative.

@jkbbwr
Copy link

@jkbbwr jkbbwr commented Dec 12, 2016

Probably related #885

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 13, 2016

I'm afraid I'm unable to replicate this failure. I thought maybe Arch was implicated because the tests are passing on Ubuntu, but now I've run the tests on Arch Linux and they also pass:

$ cat Vagrantfile 
Vagrant.configure("2") do |config|
  config.vm.box = "ogarcia/archlinux-x64"
  config.vm.provision "shell", inline: <<-SHELL
    pacman -S python-pip git python-tox --noconfirm
    git clone https://github.com/pypa/setuptools /home/vagrant/setuptools
    chown -R vagrant /home/vagrant/setuptools
    pushd /home/vagrant/setuptools
    su -c "python bootstrap.py" vagrant
  SHELL
end
$ vagrant up
Bringing machine 'default' up with 'virtualbox' provider...
==> default: Importing base box 'ogarcia/archlinux-x64'...
==> default: Matching MAC address for NAT networking...
==> default: Checking if box 'ogarcia/archlinux-x64' is up to date...
==> default: Setting the name of the VM: issue-885_default_1481658686755_61215
==> default: Clearing any previously set network interfaces...
==> default: Preparing network interfaces based on configuration...
    default: Adapter 1: nat
==> default: Forwarding ports...
    default: 22 (guest) => 2222 (host) (adapter 1)
==> default: Booting VM...
==> default: Waiting for machine to boot. This may take a few minutes...
    default: SSH address: 127.0.0.1:2222
    default: SSH username: vagrant
    default: SSH auth method: private key
    default: 
    default: Vagrant insecure key detected. Vagrant will automatically replace
    default: this with a newly generated keypair for better security.
    default: 
    default: Inserting generated public key within guest...
    default: Removing insecure key from the guest if it's present...
    default: Key inserted! Disconnecting and reconnecting using new SSH key...
==> default: Machine booted and ready!
==> default: Checking for guest additions in VM...
==> default: Mounting shared folders...
    default: /vagrant => /Users/jaraco/issue-885
==> default: Running provisioner: shell...
    default: Running: inline script
==> default: resolving dependencies...
==> default: looking for conflicting packages...
==> default: 
==> default: Packages (13) perl-error-0.17024-1  python-3.5.2-3  python-appdirs-1.4.0-4  python-packaging-16.8-1  python-pluggy-0.4.0-1  python-py-1.4.31-1  python-pyparsing-2.1.10-1  python-setuptools-1:28.8.0-1  python-six-1.10.0-2  python-virtualenv-15.0.3-1  git-2.10.2-1  python-pip-8.1.2-1  python-tox-2.5.0-1
==> default: 
==> default: Total Download Size:    25.61 MiB
==> default: Total Installed Size:  142.07 MiB
==> default: :: Proceed with installation? [Y/n] 
==> default: 
==> default: :: Retrieving packages...
==> default: downloading python-3.5.2-3-x86_64.pkg.tar.xz...
==> default: downloading python-pyparsing-2.1.10-1-any.pkg.tar.xz...
==> default: downloading python-six-1.10.0-2-any.pkg.tar.xz...
==> default: downloading python-packaging-16.8-1-any.pkg.tar.xz...
==> default: downloading python-appdirs-1.4.0-4-any.pkg.tar.xz...
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from ftp.sh.cvut.cz : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirrors.cicku.me : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.tyborek.pl : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from ftp.nluug.nl : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.hactar.xyz : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from archlinux.mirror.ba : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.neuf.no : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.archlinux.no : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.aur.rocks : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirrors.n-ix.net : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from archlinux.mirrors.uk2.net : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.f4st.host : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror5.bastelfreak.org : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirrors.dotsrc.org : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.23media.de : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.cinosure.com : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.us.leaseweb.net : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.fluxent.de : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirrors.manchester.m247.com : Could not resolve host: mirrors.manchester.m247.com
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirrors.rit.edu : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from ftp.rnl.tecnico.ulisboa.pt : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.gnomus.de : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from ftp.snt.utwente.nl : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.vfn-nrw.de : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.de.leaseweb.net : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.kaminski.io : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from archlinux.polymorf.fr : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.one.com : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from arch.midov.pl : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.js-webcoding.de : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.datacenter.by : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirrors.lug.mtu.edu : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from ftp.myrveln.se : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from archlinux.surlyjake.com : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from gluttony.sin.cvut.cz : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from ftp.halifax.rwth-aachen.de : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from fooo.biz : Connection timed out after 10000 milliseconds
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from www.gtlib.gatech.edu : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from ftp.acc.umu.se : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.pseudoform.org : The requested URL returned error: 404
==> default: error: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from pkg.adfinis-sygroup.ch : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirrors.uni-plovdiv.net : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from archlinux.beccacervello.it : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.frgl.pw : The requested URL returned error: 404
==> default: error: 
==> default: failed retrieving file 'python-setuptools-1:28.8.0-1-any.pkg.tar.xz' from mirror.united-gameserver.de : The requested URL returned error: 404
==> default: downloading python-setuptools-1:28.8.0-1-any.pkg.tar.xz...
==> default: downloading python-pip-8.1.2-1-any.pkg.tar.xz...
==> default: downloading perl-error-0.17024-1-any.pkg.tar.xz...
==> default: downloading git-2.10.2-1-x86_64.pkg.tar.xz...
==> default: downloading python-virtualenv-15.0.3-1-any.pkg.tar.xz...
==> default: downloading python-py-1.4.31-1-any.pkg.tar.xz...
==> default: downloading python-pluggy-0.4.0-1-any.pkg.tar.xz...
==> default: downloading python-tox-2.5.0-1-any.pkg.tar.xz...
==> default: checking keyring...
==> default: checking package integrity...
==> default: loading package files...
==> default: checking for file conflicts...
==> default: checking available disk space...
==> default: :: Processing package changes...
==> default: installing python...
==> default: Optional dependencies for python
==> default:     python-setuptools [pending]
==> default:     python-pip [pending]
==> default:     sqlite [installed]
==> default:     mpdecimal: for decimal
==> default:     xz: for lzma [installed]
==> default:     tk: for tkinter
==> default: installing python-pyparsing...
==> default: installing python-six...
==> default: installing python-packaging...
==> default: installing python-appdirs...
==> default: installing python-setuptools...
==> default: installing python-pip...
==> default: installing perl-error...
==> default: installing git...
==> default: Optional dependencies for git
==> default:     tk: gitk and git gui
==> default:     perl-libwww: git svn
==> default:     perl-term-readkey: git svn
==> default:     perl-mime-tools: git send-email
==> default:     perl-net-smtp-ssl: git send-email TLS support
==> default:     perl-authen-sasl: git send-email TLS support
==> default:     python2: various helper scripts
==> default:     subversion: git svn
==> default:     cvsps2: git cvsimport
==> default:     gnome-keyring: GNOME keyring credential helper
==> default: installing python-py...
==> default: installing python-virtualenv...
==> default: installing python-pluggy...
==> default: installing python-tox...
==> default: Cloning into '/home/vagrant/setuptools'...
==> default: /home/vagrant/setuptools /home/vagrant
==> default: running egg_info
==> default: writing entry points to setuptools.egg-info/entry_points.txt
==> default: writing dependency_links to setuptools.egg-info/dependency_links.txt
==> default: writing requirements to setuptools.egg-info/requires.txt
==> default: writing manifest file 'setuptools.egg-info/SOURCES.txt'
==> default: reading manifest file 'setuptools.egg-info/SOURCES.txt'
==> default: reading manifest template 'MANIFEST.in'
==> default: /usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'zip_safe'
==> default:   warnings.warn(msg)
==> default: /usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'python_requires'
==> default:   warnings.warn(msg)
==> default: /usr/lib/python3.5/distutils/dist.py:261: UserWarning: Unknown distribution option: 'setup_requires'
==> default:   warnings.warn(msg)
==> default: warning: no files found matching '*' under directory 'setuptools/_vendor'
==> default: writing manifest file 'setuptools.egg-info/SOURCES.txt'
==> default: running egg_info
==> default: writing requirements to setuptools.egg-info/requires.txt
==> default: writing entry points to setuptools.egg-info/entry_points.txt
==> default: writing setuptools.egg-info/PKG-INFO
==> default: writing dependency_links to setuptools.egg-info/dependency_links.txt
==> default: writing top-level names to setuptools.egg-info/top_level.txt
==> default: reading manifest file 'setuptools.egg-info/SOURCES.txt'
==> default: reading manifest template 'MANIFEST.in'
==> default: warning: no files found matching '*' under directory 'setuptools/_vendor'
==> default: writing manifest file 'setuptools.egg-info/SOURCES.txt'
==> default: adding minimal entry_points
==> default: Regenerating egg_info
==> default: ...and again.
$ vagrant ssh
[vagrant@arch ~]$ cd setuptools
[vagrant@arch setuptools]$ python -m tox -- -k test_namespace -v
python develop-inst-noop: /home/vagrant/setuptools
python installed: backports.unittest-mock==1.2.1,flake8==3.2.1,mccabe==0.5.2,py==1.4.31,pycodestyle==2.2.0,pyflakes==1.3.0,pytest==3.0.5,pytest-flake8==0.8.1
python runtests: PYTHONHASHSEED='1325299041'
python runtests: commands[0] | py.test -k test_namespace -v
====================================== test session starts ======================================
platform linux -- Python 3.5.2, pytest-3.0.5, py-1.4.31, pluggy-0.4.0 -- /home/vagrant/setuptools/.tox/python/bin/python
cachedir: .cache
rootdir: /home/vagrant/setuptools, inifile: pytest.ini
plugins: flake8-0.8.1, backports.unittest-mock-1.2.1
collected 220 items / 2 skipped 

setuptools/tests/test_develop.py::TestNamespaces::test_namespace_package_importable PASSED
setuptools/tests/test_namespaces.py::TestNamespaces::test_mixed_site_and_non_site PASSED
setuptools/tests/test_namespaces.py::TestNamespaces::test_pkg_resources_import PASSED

===================================== 217 tests deselected ======================================
====================== 3 passed, 2 skipped, 217 deselected in 5.52 seconds ======================
____________________________________________ summary ____________________________________________
  python: commands succeeded
  congratulations :)

What does one have to do to replicate the failure you've seen?

@felixonmars
Copy link
Contributor Author

@felixonmars felixonmars commented Dec 22, 2016

Sorry for the late response. I am using python setup.py ptr to run the tests, looks like this has something to do with the problem...

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 22, 2016

I'm able to replicate the issue as long as I have an old setuptools (<=31.0.0) installed in the environment.

$ python -m rwt pytest-runner pytest -- setup.py ptr --addopts='-k test_namespace'
Collecting pytest-runner
  Using cached pytest_runner-2.9-py2.py3-none-any.whl
Collecting pytest
  Using cached pytest-3.0.5-py2.py3-none-any.whl
Collecting py>=1.4.29 (from pytest)
  Using cached py-1.4.32-py2.py3-none-any.whl
Installing collected packages: pytest-runner, py, pytest
Successfully installed py-1.4.32 pytest-3.0.5 pytest-runner-2.9
running ptr
running egg_info
writing setuptools.egg-info/PKG-INFO
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
writing requirements to setuptools.egg-info/requires.txt
writing top-level names to setuptools.egg-info/top_level.txt
reading manifest file 'setuptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'setuptools/_vendor'
writing manifest file 'setuptools.egg-info/SOURCES.txt'
running build_ext
====================================== test session starts =======================================
platform darwin -- Python 3.6.0rc2, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
rootdir: /Users/jaraco/Dropbox/code/main/setuptools, inifile: pytest.ini
collected 221 items / 2 skipped 

setuptools/tests/test_develop.py .
setuptools/tests/test_namespaces.py ..F

============================================ FAILURES ============================================
____________________ TestNamespaces.test_namespace_package_installed_and_cwd _____________________

self = <setuptools.tests.test_namespaces.TestNamespaces object at 0x10660cac8>
tmpdir = local('/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-14/test_namespace_package_install0')

    @pytest.mark.skipif(bool(os.environ.get("APPVEYOR")),
        reason="https://github.com/pypa/setuptools/issues/851")
    def test_namespace_package_installed_and_cwd(self, tmpdir):
        """
            Installing a namespace packages but also having it in the current
            working directory, only one version should take precedence.
            """
        pkg_A = namespaces.build_namespace_package(tmpdir, 'myns.pkgA')
        target = tmpdir / 'packages'
        # use pip to install to the target directory
        install_cmd = [
            'pip',
            'install',
            str(pkg_A),
            '-t', str(target),
        ]
        subprocess.check_call(install_cmd)
        namespaces.make_site_dir(target)
    
        # ensure that package imports and pkg_resources imports
        pkg_resources_imp = [
            sys.executable,
            '-c', 'import pkg_resources; import myns.pkgA',
        ]
        env = dict(PYTHONPATH=str(target))
>       subprocess.check_call(pkg_resources_imp, env=env, cwd=str(pkg_A))

setuptools/tests/test_namespaces.py:104: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

popenargs = (['/Library/Frameworks/Python.framework/Versions/3.6/bin/python3', '-c', 'import pkg_resources; import myns.pkgA'],)
kwargs = {'cwd': '/private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-14/test_namespace_package_in.../var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-14/test_namespace_package_install0/packages'}}
retcode = 1
cmd = ['/Library/Frameworks/Python.framework/Versions/3.6/bin/python3', '-c', 'import pkg_resources; import myns.pkgA']

    def check_call(*popenargs, **kwargs):
        """Run command with arguments.  Wait for command to complete.  If
        the exit code was zero then return, otherwise raise
        CalledProcessError.  The CalledProcessError object will have the
        return code in the returncode attribute.
    
        The arguments are the same as for the call function.  Example:
    
        check_call(["ls", "-l"])
        """
        retcode = call(*popenargs, **kwargs)
        if retcode:
            cmd = kwargs.get("args")
            if cmd is None:
                cmd = popenargs[0]
>           raise CalledProcessError(retcode, cmd)
E           subprocess.CalledProcessError: Command '['/Library/Frameworks/Python.framework/Versions/3.6/bin/python3', '-c', 'import pkg_resources; import myns.pkgA']' returned non-zero exit status 1.

/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py:291: CalledProcessError
-------------------------------------- Captured stdout call --------------------------------------
Processing /private/var/folders/c6/v7hnmq453xb6p2dbz1gqc6rr0000gn/T/pytest-of-jaraco/pytest-14/test_namespace_package_install0/myns.pkgA
Installing collected packages: myns.pkgA
  Running setup.py install for myns.pkgA: started
    Running setup.py install for myns.pkgA: finished with status 'done'
Successfully installed myns.pkgA-1.0
-------------------------------------- Captured stderr call --------------------------------------
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3015, in <module>
    @_call_aside
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2999, in _call_aside
    f(*args, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3043, in _initialize_master_working_set
    for dist in working_set
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 3043, in <genexpr>
    for dist in working_set
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2573, in activate
    declare_namespace(pkg)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2147, in declare_namespace
    _handle_ns(packageName, path_item)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2087, in _handle_ns
    _rebuild_mod_path(path, packageName, module)
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2116, in _rebuild_mod_path
    orig_path.sort(key=position_in_sys_path)
AttributeError: '_NamespacePath' object has no attribute 'sort'
====================================== 217 tests deselected ======================================
================= 1 failed, 3 passed, 2 skipped, 217 deselected in 8.93 seconds ==================
$ 

However, if I upgrade my python environment setuptools to 31.0.1, the issue goes away:

$ python -m pip install setuptools==31.0.1
Collecting setuptools==31.0.1
  Using cached setuptools-31.0.1-py2.py3-none-any.whl
Installing collected packages: setuptools
  Found existing installation: setuptools 31.0.0
    Uninstalling setuptools-31.0.0:
      Successfully uninstalled setuptools-31.0.0
Successfully installed setuptools-31.0.1
$ python -m rwt pytest-runner pytest -- setup.py ptr --addopts='-k test_namespace'
Collecting pytest-runner
  Using cached pytest_runner-2.9-py2.py3-none-any.whl
Collecting pytest
  Using cached pytest-3.0.5-py2.py3-none-any.whl
Collecting py>=1.4.29 (from pytest)
  Using cached py-1.4.32-py2.py3-none-any.whl
Installing collected packages: pytest-runner, py, pytest
Successfully installed py-1.4.32 pytest-3.0.5 pytest-runner-2.9
running ptr
running egg_info
writing setuptools.egg-info/PKG-INFO
writing dependency_links to setuptools.egg-info/dependency_links.txt
writing entry points to setuptools.egg-info/entry_points.txt
writing requirements to setuptools.egg-info/requires.txt
writing top-level names to setuptools.egg-info/top_level.txt
reading manifest file 'setuptools.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*' under directory 'setuptools/_vendor'
writing manifest file 'setuptools.egg-info/SOURCES.txt'
running build_ext
====================================== test session starts =======================================
platform darwin -- Python 3.6.0rc2, pytest-3.0.5, py-1.4.32, pluggy-0.4.0
rootdir: /Users/jaraco/Dropbox/code/main/setuptools, inifile: pytest.ini
collected 221 items / 2 skipped 

setuptools/tests/test_develop.py .
setuptools/tests/test_namespaces.py ...

====================================== 217 tests deselected ======================================
====================== 4 passed, 2 skipped, 217 deselected in 8.84 seconds =======================
$ 

The issue seems to be that when that test runs, it is somehow picking up the system-installed setuptools and not the version under test. That doesn't happen in the canonical tests because they're run under tox now, which installs setuptools rather than setup.py ptr, which relies on the cwd to import the package.

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 22, 2016

Actually, now I notice that my failure is different from yours, but still I suspect the failing test is dependent on which setuptools is installed.

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 22, 2016

Indeed, if I downgrade to setuptools<29 in my system python, I additionally get the test failure you got.

@felixonmars
Copy link
Contributor Author

@felixonmars felixonmars commented Dec 22, 2016

I also got your errors now, but the errors in the OP still persist.

I have tried to install the latest setuptools to system and confirmed that fixes the problem.

jaraco added a commit that referenced this issue Dec 22, 2016
…etuptools under test. Fixes #884.
@jaraco
Copy link
Member

@jaraco jaraco commented Dec 22, 2016

So I've devised a workaround, pushed to the issue-884 branch. I'm trying to decide if setuptools should support running tests in this model (with pytest-runner) or if the added complication to the test code plus other issues that might arise down the road suggest that this should be an unsupported test runner, and that tests should always be run using tox or with the current setuptools installed into the environment.

Perhaps you have an opinion on the matter? What leads you to use pytest-runner over tox? Would a suitable workaround be to 'setup.py develop' setuptools before running 'ptr'? One of the reasons I've been moving away from pytest-runner is exactly this one, where subprocesses don't get the "package" environment (mainly sys.path) of the parent process without the tests being aware of this contingency. If pytest-runner were the recommended test runner for setuptools, it would certainly be the case that setuptools' test suite should support this mode, but since tox is the recommended runner, I'm inclined not to support the nuances of pytest-runner. I'm open to being convinced if you feel strongly about it.

@felixonmars
Copy link
Contributor Author

@felixonmars felixonmars commented Dec 22, 2016

The main reason is that generally we don't allow network access during the whole build/test process when packaging for a distro. I have tried tox --sitepackages and the generated errors look even harder to fix (maybe I am missing something here?). I didn't yet try the setup.py develop workaround, and will give it a try now.

@felixonmars
Copy link
Contributor Author

@felixonmars felixonmars commented Dec 22, 2016

Hrm, the setup.py develop workaround doesn't seem to work here, as it writes to system root (read-only for a build chroot) or relies on PYTHONPATH (for --install-dir) which should bring back the problem.

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 22, 2016

I see. Yes, that is a meaningful test case which seems reasonable to support.

I think I've found how to make it work. As part of your bootstrapping logic, grab the test dependencies:

$ pip download -r tests/requirements.txt -d test-deps
$ pip install tox

Then when offline, invoke the tests thus:

$ PIP_FIND_LINKS=test-deps tox

That worked for me here on macOS. Will that technique also work for your use case?

@felixonmars
Copy link
Contributor Author

@felixonmars felixonmars commented Dec 22, 2016

Since we do not have a network-ready bootstrap process (which brings in nondeterminacy and not good for reproducible builds), I can only manually list all wheels down in the dependency tree as sources and update them for each version.

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 22, 2016

I'm leaning now toward this being a pytest-runner bug - that pytest-runner should set PYTHONPATH to include the eggs added to sys.path. Oh, I see in pytest-dev/pytest-runner#20, that was done, but it doesn't include the current working directory. Then, as I delve into the test command, I see that it already has support for adding the project itself to sys.path.

So maybe the issue is more straightforward. Maybe the tests are failing because they're setting PYTHONPATH rather than extending it.

@jaraco
Copy link
Member

@jaraco jaraco commented Dec 22, 2016

I had a breakthrough! Turns out all of the behavior that was needed was already coded in the tests command. I just had forgotten I'd written it. This next patch is cleaner, adds minimal complication, and addresses the issue.

@jaraco jaraco closed this in 7a1c700 Dec 22, 2016
@felixonmars
Copy link
Contributor Author

@felixonmars felixonmars commented Dec 22, 2016

That's awesome! Thanks a lot for the additional effort :D

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants