New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setuptools 34.0.2 breaks new virtual envs with six < 1.10 #947

Closed
truthdoug opened this Issue Jan 26, 2017 · 4 comments

Comments

Projects
None yet
3 participants
@truthdoug

truthdoug commented Jan 26, 2017

The recent change to strip out vendored packages (like six) breaks projects that pip install a version of six earlier than 1.10

This was also reported as an issue on virtualenv with code examples showing the problem. The commit on setuptools appears to be the root cause.

@jaraco

This comment has been minimized.

Member

jaraco commented Jan 28, 2017

This change to setuptools was made intentionally in order to avoid the ugly hacks and complicated bootstrapping scenarios imposed by such hacks.

The issue here is that the offending packages that are requiring old versions of six. They simply are no longer compatible with setuptools.

Those failing packages accepted the burden of incompatibility with any package that declared a later six dependency when they pinned their dependency to a specific version. Since those packages are demanding an old version of six, then it should be those packages that either vendor six themselves or adapt to support newer versions of six.

I did look into the possibility of expanding the supported six versions in setuptools. If I change the version to 1.6.0, some integration tests start to fail (because stevedore depends on six 1.9), but otherwise the tests pass. Perhaps setting a lower lower bound will be viable and save some headaches.

@jaraco jaraco closed this in edfa868 Jan 28, 2017

@asottile

This comment has been minimized.

asottile commented Feb 2, 2017

In applications, it is a best practice to pin every dependency with == in requirements.txt for reproducibility.

This usually means pip freeze --all > requirements.txt.

If one were to generate a requirements.txt with an older version of six / pyparsing / etc. they're now suddenly broken as virtualenv downloads the newest version of setuptools (demonstrated in pypa/virtualenv#1009).

@jaraco

This comment has been minimized.

Member

jaraco commented Feb 2, 2017

@asottile: How do you suggest these inherent dependency conflicts be reconciled?

@asottile

This comment has been minimized.

asottile commented Feb 2, 2017

That's a good question!

In an ideal world, pip wouldn't upgrade setuptools to a version that conflicts with already installed dependencies (and would do dependency resolution) but I don't think that's really something that's doable without making pip way more intelligent and way slower.

It's possible this is just an "unfortunate" combination of two separate "unfortunate" decisions, notably:

  • virtualenv upgrades pip / setuptools / wheel by default (new behaviour in 15)
  • setuptools unvendored dependencies (new in 34)

At least personally, the first behaviour is alarming and makes it very difficult to have repeatable builds (without scattering --no-download, VIRTUALENV_NO_DOWNLOAD=1 or PIP_INDEX_URL=... virtualenv all over our ~thousands of codebases). The second seems not a problem to me, I rather like that setuptools now acts more like a normal python package (though the pypa tools don't treat it as such: magically upgrading, excluding from freeze, etc.).

In short, big shrugs -- it might just be "unfortunate: won't fix" and that would be ok too!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment