Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vendoring logic subtly breaks later usage of unvendored system versions after importing pkg_resources.extern #1383

Open
eli-schwartz opened this issue May 27, 2018 · 5 comments

Comments

@eli-schwartz
Copy link

@eli-schwartz eli-schwartz commented May 27, 2018

In theory, if pkg_resources.extern is devendored, it should defer to the system installation of e.g. packaging, which means that:

>>> packaging.version
<module 'pip._vendor.packaging.version' from '/usr/lib/python3.6/site-packages/packaging/version.py'>
>>> pkg_resources.extern.packaging.version
<module 'pip._vendor.packaging.version' from '/usr/lib/python3.6/site-packages/packaging/version.py'>

It would be expected that I could then compare the results, since they're the same thing.

>>> pkg_resources.extern.packaging.version.parse('3.0') == packaging.version.parse('2.0')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '>' not supported between instances of 'Version' and 'Version'

Oops?

For comparison, this works fine when using the similarly devendored pip.

>>> pip._vendor.packaging.version
<module 'packaging.version' from '/usr/lib/python3.6/site-packages/packaging/version.py'>
>>> packaging.version
<module 'packaging.version' from '/usr/lib/python3.6/site-packages/packaging/version.py'>
>>> pip._vendor.packaging.version.parse('3.0') > packaging.version.parse('2.0')
True

Note that pkg_resources must not be devendored from pip, or else it must be deleted from vendor/__init__.py, since otherwise it will be automatically imported. Even importing pkg_resources causes pip to fail as well.

>>> import pip._vendor.packaging.version
>>> import packaging.version
>>> pip._vendor.packaging.version.parse('3.0') > packaging.version.parse('2.0')
True

vs.

>>> import pip._vendor.packaging.version
>>> import pkg_resources.extern
>>> import packaging.version
>>> pip._vendor.packaging.version.parse('3.0') > packaging.version.parse('2.0')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: '>' not supported between instances of 'Version' and 'Version'

pypa/pip#5429 (comment) suggested the problem is deleting the global module unconditionally, whenever something vendored is in use. This strikes me as incredibly dangerous.

@yan12125
Copy link

@yan12125 yan12125 commented May 27, 2018

I believe issue #580 is related. Seems setuptools maintainers were considering switching to the pip approach regarding vendoring to fix such issues.

@jaraco
Copy link
Member

@jaraco jaraco commented Jul 9, 2018

The origin of that hack is 6a5145a. Even though I wrote that code and I've stared at it for at least 15 minutes, I don't fully understand what's going on there. It seems plausible that the hack is over-reaching and really should only be employed when there's a prefix. So let's give that patch a go.

@jaraco jaraco closed this in 949bd16 Jul 9, 2018
@jaraco
Copy link
Member

@jaraco jaraco commented Jul 10, 2018

In the commit, @richard-scott reports that the patch (subtly altered) didn't fix the issue.

What we could use at this point is a way to replicate the issue. Could someone devise a recipe that replicates the issue (preferably in a virtualenv and cross-platform or in a docker image).

@jaraco jaraco reopened this Jul 10, 2018
@richard-scott
Copy link

@richard-scott richard-scott commented Jul 10, 2018

All I did was install a fresh Ubuntu server and attempt to list outdated packages:

$ pip3 list --no-cache-dir --disable-pip-version-check --outdated --format=freeze

That said, I've removed the fix and the above command now works. My setup does automatically patch any pip packages on boot, so one of those may have been updated.

I'm currently running pip-10.0.1, and setuptools-40.0.0.

@yan12125
Copy link

@yan12125 yan12125 commented Jul 29, 2019

I'm currently running pip-10.0.1, and setuptools-40.0.0.

@richard-scott This might be the reason for the breakage. You need setuptools >= 40.0.0 and pip >= 19.1 to get this command working with unvendored pip.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.