"pip install --upgrade distribute" fails with Python 3 #650

Closed
regebro opened this Issue Aug 23, 2012 · 24 comments

Projects

None yet

10 participants

@regebro
regebro commented Aug 23, 2012

Essentially, since pip uses distribute to do the upgrade, the "import distribute" statements in the pip-provided scripts to gather egg_info and install will end up importing it from the downloaded package instead of from the system packages. But that code is Python 2, and you get syntax errors:

from setuptools.dist import _get_unpatched

File "setuptools\dist.py", line 103

except ValueError, e:
                 ^

SyntaxError: invalid syntax


Command python setup.py egg_info failed with error code 1 in C:\Users\root\AppData\Local\Temp\pip\build\distribute

Now, I found solutions for this, but they break everything else (see regebro/pip@4c11aaa ) so I'm not sure this is fixable without a lot of special casing for Distribute. Seeing as this bug only can appear in Python 3 with packages that have a module called "setuptools" it's a pretty marginal edge case. Perhaps just noting that you can't upgrade Distribute with pip under Python 3 but will have to reinstall it instead is an acceptable solution?

Contributor
carljm commented Aug 25, 2012

Yes, the import setuptools in the installation subprocess command is pretty critical to pip :-)

I'd certainly be happy with a docs note, at least as a stopgap until/unless someone is motivated enough to try to find a better fix. I've often wished there was a command-line flag to invoke python without the current-directory-on-sys.path behavior.

Member
dholth commented Aug 29, 2012

In the meantime can we avoid uninstalling distribute during an upgrade? Things get messy with --ignore-installed and packages requiring distribute.

Contributor
carljm commented Aug 29, 2012

I'm not sure how avoiding uninstalling distribute would fix this problem?

Member
dholth commented Aug 29, 2012

With some packages that list distribute as a req, pip tries to upgrade distribute, uninstalls the older one, fails, and leaves a broken virtualenv. Possibly mainly with my wheel patches.

On Aug 29, 2012, at 4:52 PM, Carl Meyer notifications@github.com wrote:

I'm not sure how avoiding uninstalling distribute would fix this problem?


Reply to this email directly or view it on GitHub.

Contributor
carljm commented Aug 29, 2012

Ah, I see. Yeah, a patch to make pip ignore distribute as a
dependency-to-be-upgraded (at least on Py3k) seems like a reasonable
temporary workaround for those cases.

Member
dholth commented Aug 30, 2012

How about a general-purpose --ignore=name (although --ignore is taken as short for --ignore-installed)

Contributor
pnasrat commented Aug 30, 2012

--exclude or something.

On 30 August 2012 14:44, dholth notifications@github.com wrote:

How about a general-purpose --ignore=name (although --ignore is taken as
short for --ignore-installed)


Reply to this email directly or view it on GitHubhttps://github.com/pypa/pip/issues/650#issuecomment-8159745.

Contributor
carljm commented Aug 30, 2012

I'm not opposed to the --exclude idea. Should get its own ticket.

Member
dholth commented Sep 4, 2012

Avoid a lot of pain by adding to pip/req.py:

for requirement in to_install:

if requirement.name == "distribute": logger.notify(...); continue

Member
dholth commented Sep 4, 2012

Fix for non-2to3 distribute upgrading problems: dholth@341e71d

Member
dholth commented Sep 4, 2012

How does the fix break everything else?

Contributor
pnasrat commented Sep 6, 2012

@dholth not sure of the context of your last comment - can you clarify.

Member
dholth commented Sep 6, 2012

Last para of bug description

On Sep 6, 2012, at 2:24 AM, Paul Nasrat notifications@github.com wrote:

@dholth not sure of the context of your last comment - can you clarify.


Reply to this email directly or view it on GitHub.

Perhaps just noting that you can't upgrade Distribute with pip under Python 3 but will have to reinstall it instead is an acceptable solution?

Having this issue when running pip3 install --upgrade distribute, how can I do it to upgrade distribute?

I'm curious as well to how the attempted fix above "breaks everything else"

Also, an excerpt:

install_args = [
    sys.executable, '-c',
-   "import setuptools;__file__=%r;"\
+   "__file__=%r;"\
    "exec(compile(open(__file__).read().replace('\\r\\n', '\\n'), __file__, 'exec'))" % self.setup_py] +\
    list(global_options) + [
    'install',

Is that import that was removed ever required?

Contributor
carljm commented Nov 25, 2012

That import is quite important, yes, and removing it is what "breaks everything else". Setuptools monkeypatches the distutils framework when it is imported to provide some additional features, and forcing the import of it before executing setup.py is a core pip feature; it's what allows pip to provide uniform handling of projects that do and don't use setuptools in their setup.py. With that import removed, "pip install" will break for any project that has "from distutils.core import setup" rather than "from setuptools import setup" in its setup.py.

Member
pfmoore commented Jan 8, 2013

At a minimum I would hope that it's possible to do at least some of the following:

  • add some comments to the documentation
  • allow the user to specify that pip should never upgrade distribute (issue #654)
  • never attempt to upgrade distribute due to implicit dependency resolution (i.e., the user has to explicitly specify it on the command line)
  • raise an error with a better explanation of the issue if the user does attempt to manually upgrade distribute with pip
@florentx florentx referenced this issue in PyCQA/pycodestyle Feb 25, 2013
Closed

upgrade is broken on Python 3 #172

In addition to @pfmoore's list, I'd appreciate if pip freeze excluded distribute from the list of installed packages. That would solve the issues I've run into bringing up new developers' virtualenvs on my projects. I'm encountering this issue in purely Python 2.6/2.7 environments; attempting to upgrade distribute with pip seems like bad news regardless of Python version.

Member
dholth commented Mar 22, 2013

We've had features in either bugs or testing that just skip trying to install distribute or setuptools at all... so pip says "I can't do that Dave" instead of "crash".

Member
dholth commented Mar 22, 2013

the install side is the right place to solve this. Or use grep -v

Jamey Sharp notifications@github.com wrote:

In addition to @pfmoore's list, I'd appreciate if pip freeze excluded distribute from the list of installed packages. That would solve the issues I've run into bringing up new developers' virtualenvs on my projects. I'm encountering this issue in purely Python 2.6/2.7 environments; attempting to upgrade distribute with pip seems like bad news regardless of Python version.


Reply to this email directly or view it on GitHub.

As a workaround, is it reasonable to use easy_install instead of pip to upgrade distribute? I just did that and nothing seems to have broken in any obvious way.

regebro commented May 2, 2013

Absolutely. I don't know if you keep the possibility to uninstall it from pip, but hey, it's distribute. You need it. :-)

This is not an important bug in any way, it's just annoying, and it keeps cropping up because people run into it and post new bug reports. :-)

It may not be an important bug in pip, but as you say, people keep running into it because they want to upgrade some package that happen to explicitly depend on distribute. That makes it pretty important for the users because it breaks the normal workflow.

Contributor
qwcode commented Jul 14, 2013

with the release of distribute-0.7.3 (and setuptools>=0.8 which is py2/py3 compatible code), this is no longer an issue.

pip install --upgrade distribute works in py3

@qwcode qwcode closed this Jul 14, 2013
@Zincr0 Zincr0 added a commit to Zincr0/pyscrap3 that referenced this issue Mar 31, 2014
@Zincr0 Zincr0 removed distribute dependency (bug pypa/pip#650) 7ae425b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment