New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation issues on Travis CI #345

Closed
bb-migration opened this Issue Feb 12, 2015 · 15 comments

Comments

Projects
None yet
1 participant
@bb-migration

bb-migration commented Feb 12, 2015

Originally reported by: Anonymous


Since the upload of version setuptools version 12.1 yesterday, our builds on Travis start to fail:
https://travis-ci.org/bioidiap/bob.core/jobs/50389380

Using the latest bootstrap-buildout.py, we get the following error message:

$ python bootstrap-buildout.py
<string>:144: DeprecationWarning: `use_setuptools` is deprecated. To enforce a specific version of setuptools, use `pkg_resources.require`.
Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-12.1.zip
Extracting in /tmp/tmpRCrQju
Now working in /tmp/tmpRCrQju/setuptools-12.1
Building a Setuptools egg in /tmp/tmp7CIXgd
/tmp/tmpRCrQju/setuptools-12.1/setuptools/command/egg_info.py:171: DeprecationWarning: Parameters to load are deprecated. Call .resolve and .require separately.
writer = ep.load(installer=installer)
/tmp/tmp7CIXgd/setuptools-12.1-py2.6.egg
Traceback (most recent call last):
File "bootstrap-buildout.py", line 98, in <module>
ez['use_setuptools'](**setup_args)
File "<string>", line 174, in use_setuptools
File "<string>", line 129, in _do_download
File "build/bdist.linux-x86_64/egg/setuptools/__init__.py", line 11, in <module>
File "build/bdist.linux-x86_64/egg/setuptools/extension.py", line 8, in <module>
File "build/bdist.linux-x86_64/egg/setuptools/dist.py", line 19, in <module>
File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 83, in <module>
AttributeError: 'module' object has no attribute '_vendor'

Interestingly, lines 81 and 82 of pkg_resources/__init__.py (the imports of the _vendor module) pass, but not the actual use of it.
Sorry for not being able to supply more information...

Could you please double-check, what might be going on?

Thank you
Manuel


@bb-migration

This comment has been minimized.

bb-migration commented Feb 12, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


The changes between 12.0.5 and 12.1 were trivially small. I would be surprised if the 12.1 release was a regression from 12.0.5. Do you have reason to believe that (a) the issue did not exist with setuptools 12.0.5 and (b) the bootstrap-buildout has not changed?

@bb-migration

This comment has been minimized.

bb-migration commented Feb 12, 2015

Original comment by Anonymous:


I am not sure about 12.0.5, but I am sure that 12.0.4 was working: https://travis-ci.org/bioidiap/bob.core/jobs/48035636

@bb-migration

This comment has been minimized.

bb-migration commented Feb 12, 2015

Original comment by Anonymous:


And I am sure that I haven't changed the bootstrap-buildout.py file (which is still up to date, I have double-checked that).

On the other hand, I am not 100% sure that Travis hasn't updated their system, though I didn't receive a notification...

@bb-migration

This comment has been minimized.

bb-migration commented Feb 14, 2015

Original comment by znerol (Bitbucket: znerol, GitHub: znerol):


Related issue on travis issue tracker.

@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by Anonymous:


Indeed, it seems to be an issue with the pre-installed setuptools, which I can reproduce locally.
When using a virtualenv system with pre-installed setuptools version 11.3.1 (in my case) or 12.0.5 (on travis), I get the error:

#!sh
$ /path/to/virtual/system/python bootstrap-buildout.py 
Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-12.1.zip
Extracting in /tmp/tmpuCQhNZ
Now working in /tmp/tmpuCQhNZ/setuptools-12.1
Building a Setuptools egg in /tmp/tmpdBboBt
/tmp/tmpdBboBt/setuptools-12.1-py2.7.egg
Traceback (most recent call last):
  File "bootstrap-buildout.py", line 98, in <module>
    ez['use_setuptools'](**setup_args)
  File "<string>", line 174, in use_setuptools
  File "<string>", line 129, in _do_download
  File "build/bdist.linux-x86_64/egg/setuptools/__init__.py", line 11, in <module>
  File "build/bdist.linux-x86_64/egg/setuptools/extension.py", line 8, in <module>
  File "build/bdist.linux-x86_64/egg/setuptools/dist.py", line 19, in <module>
  File "build/bdist.linux-x86_64/egg/pkg_resources/__init__.py", line 83, in <module>
AttributeError: 'module' object has no attribute '_vendor'

So far, my workaround is to use the --setuptools-version option selecting exactly the version that we have:

#!sh
$ /path/to/virtual/system/python bootstrap-buildout.py --setuptools-version 11.3.1

However, we would need to update the travis configuration files each time that they update their system. Since we have more than 40 packages:
https://github.com/idiap/bob/wiki/Packages
you can imaging that I don't want to do that.

The solution in the travis issue tracker would probably work as well, but as it is written there, this is a work-around and no fix. Again, I don't know how long this solution will work.

@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


The issue reported in the travis bug tracker is slightly different, displaying a VersionConflict prior to the AttributeError:

$ python bootstrap.py
Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-12.1.zip
Extracting in /tmp/tmpeqvl0x
Now working in /tmp/tmpeqvl0x/setuptools-12.1
Building a Setuptools egg in /tmp/tmp4uaaq8
/tmp/tmp4uaaq8/setuptools-12.1-py3.3.egg
Traceback (most recent call last):
File "<string>", line 162, in use_setuptools
File "/home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages/pkg_resources/__init__.py", line 918, in require
needed = self.resolve(parse_requirements(requirements))
File "/home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages/pkg_resources/__init__.py", line 810, in resolve
raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (setuptools 12.0.5 (/home/travis/virtualenv/python3.3.5/lib/python3.3/site-packages), Requirement.parse('setuptools>=12.1'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "bootstrap.py", line 92, in <module>
ez['use_setuptools'](**setup_args)
File "<string>", line 174, in use_setuptools
File "<string>", line 129, in _do_download
File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
File "/tmp/tmp4uaaq8/setuptools-12.1-py3.3.egg/setuptools/__init__.py", line 11, in <module>
File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
File "/tmp/tmp4uaaq8/setuptools-12.1-py3.3.egg/setuptools/extension.py", line 8, in <module>
File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
File "/tmp/tmp4uaaq8/setuptools-12.1-py3.3.egg/setuptools/dist.py", line 19, in <module>
File "<frozen importlib._bootstrap>", line 1565, in _find_and_load
File "<frozen importlib._bootstrap>", line 1532, in _find_and_load_unlocked
File "/tmp/tmp4uaaq8/setuptools-12.1-py3.3.egg/pkg_resources/__init__.py", line 83, in <module>
AttributeError: 'module' object has no attribute '_vendor'
The command "python bootstrap.py" failed and exited with 1 during .
@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Aha. The reason that both tracebacks appear in the above report is because it's on Python 3, so the underlying exception isn't swallowed by the subsequent exception. Good work Python 3!

@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


I think I now know what's happening. In ez_setup.use_setuptools, which is invoked by buildout's bootstrap, the code will use pkg_resources to determine if the installed setuptools is a suitable version. If it is not, it unloads pkg_resources (using del pkg_resources, sys.modules['pkg_resources']). This technique worked before pkg_resources got subpackages (namely _vendor). Now, when the VersionConflict occurs, pkg_resources is unloaded from sys.modules, but pkg_resources._vendor.* is not, so when pkg_resources is reloaded, and the conditional import takes place, the _vendor modules are already in sys.modules, so they're not added to the newly-imported pkg_resources namespace.

@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Unload all pkg_resources modules and not just the main module. Fixes #345.

@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


Extract function for unloading pkg_resources. Ref #345.

@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by jaraco (Bitbucket: jaraco, GitHub: jaraco):


The fix was released as 12.2. Can you confirm the fix in your test runs?

@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by Anonymous:


I am currently re-running the tests. I'll let you know.

@bb-migration

This comment has been minimized.

bb-migration commented Feb 16, 2015

Original comment by Anonymous:


Yes, I can confirm that the tests are running now, both on Travis and using the local virtualenv.

Thank you very much for fixing this issue!
Manuel

@bb-migration

This comment has been minimized.

bb-migration commented Feb 17, 2015

Original comment by znerol (Bitbucket: znerol, GitHub: znerol):


Thanks, that's also working for me.

@bb-migration

This comment has been minimized.

bb-migration commented Feb 17, 2015

Original comment by tseaver (Bitbucket: tseaver, GitHub: tseaver):


Clears up buildout/buildout#237, too. Thanks, @jaraco!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment