Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip fail after upgrade to 1.5.5 #1800

Closed
robsonpeixoto opened this issue May 7, 2014 · 30 comments
Closed

pip fail after upgrade to 1.5.5 #1800

robsonpeixoto opened this issue May 7, 2014 · 30 comments
Labels
auto-locked Outdated issues that have been locked by automation C: wheel The wheel format and 'pip wheel' command

Comments

@robsonpeixoto
Copy link

$ pip install --upgrade pip
Downloading/unpacking pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-1.5.5-py2.py3-none-any.whl#md5=03a932d6f82a3887d8de1cdb837c87ed
  Using download cache from /Users/robinho/Library/Caches/pip-downloads/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fpy2.py3%2Fp%2Fpip%2Fpip-1.5.5-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 1.5.4
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
$ pip
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 2749, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 446, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 459, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 628, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pip==1.5.4

Using a python installed by homebrew.

$ python --version
Python 2.7.6
$ brew --version
0.9.5
$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.9.2
BuildVersion:   13C1021
@dstufft
Copy link
Member

dstufft commented May 7, 2014

Hmm, for some reason your pip command wasn't updated. I wasn't able to reproduce with a quick attempt here locally. You could try do python -m pip --upgrade --force-reinstall pip to see if that fixes it.

@robsonpeixoto
Copy link
Author

@dstufft don't work.

$ python -m pip install --upgrade --force-reinstall pip
Downloading/unpacking pip
  Using download cache from /Users/robinho/Library/Caches/pip-downloads/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fpy2.py3%2Fp%2Fpip%2Fpip-1.5.5-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 1.5.5
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...
$ pip
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 2749, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 446, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 459, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 628, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pip==1.5.4

Maybe it is a bug on homebrew install =/. What's your opinion about it?

$ mkvirtualenv test
New python executable in test/bin/python2.7
Also creating executable in test/bin/python
Installing setuptools, pip...done.

$ pip list --outdated
pip (Current: 1.5.4 Latest: 1.5.5)
setuptools (Current: 2.2 Latest: 3.5.1)

$ pip install --upgrade pip
Downloading/unpacking pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-1.5.5-py2.py3-none-any.whl#md5=03a932d6f82a3887d8de1cdb837c87ed
  Using download cache from /Users/robinho/Library/Caches/pip-downloads/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fpy2.py3%2Fp%2Fpip%2Fpip-1.5.5-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 1.5.4
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...

$ pip --version
pip 1.5.5 from /Users/robinho/.virtualenvs/test/lib/python2.7/site-packages (python 2.7)

@dstufft
Copy link
Member

dstufft commented May 7, 2014

What happens if you do which pip?

@robsonpeixoto
Copy link
Author

Before upgrade:

$ which pip
/usr/local/bin/pip

$ pip --version
pip 1.5.4 from /usr/local/Cellar/python/2.7.6_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.5.4-py2.7.egg (python 2.7)

After upgrade:

$ pip install --upgrade pip
Downloading/unpacking pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-1.5.5-py2.py3-none-any.whl#md5=03a932d6f82a3887d8de1cdb837c87ed
  Using download cache from /Users/robinho/Library/Caches/pip-downloads/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fpy2.py3%2Fp%2Fpip%2Fpip-1.5.5-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 1.5.4
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...

$ which pip
/usr/local/bin/pip

$ pip --version
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 2749, in <module>
    working_set = WorkingSet._build_master()
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 446, in _build_master
    return cls._build_from_requirements(__requires__)
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 459, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 628, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pip==1.5.4

@Ivoz
Copy link
Contributor

Ivoz commented May 7, 2014

Either your pythons and paths are horribly messed up, or this might just be an old distribute/setuptools blowup.

Try these:

# Get rid of them
$ python -m pip uninstall -y setuptools
$ python -m pip uninstall -y distribute
# the next command should FAIL now because the above should be uninstalled
$ python -c "import pkg_resources"
# reinstall stuff now
$ python -m pip install -U --force-reinstall setuptools
$ python -m pip install -U --force-reinstall pip
# now check things work and are the same
$ python -m pip --version
$ pip --version
$ pip list

@myint
Copy link
Contributor

myint commented May 7, 2014

Does pip try to check/upgrade the setuptools version to avoid these situations?

@robsonpeixoto
Copy link
Author

Not work. I don't known why, but the uninstall keep the pkg_resources.

$ python -m pip uninstall -y setuptools
Uninstalling setuptools:
  Successfully uninstalled setuptools

$ python -m pip uninstall -y distribute
Cannot uninstall requirement distribute, not installed
Storing debug log for failure in /Users/robinho/.pip/pip.log

$ python -c "import pkg_resources"

$ python -m pip install -U --force-reinstall setuptools
Downloading/unpacking setuptools
  Using download cache from /Users/robinho/Library/Caches/pip-downloads/https%3A%2F%2Fpypi.python.org%2Fpackages%2F3.4%2Fs%2Fsetuptools%2Fsetuptools-3.6-py2.py3-none-any.whl
Installing collected packages: setuptools
Successfully installed setuptools
Cleaning up...
$ python -m pip install -U --force-reinstall pip
Downloading/unpacking pip
  Using download cache from /Users/robinho/Library/Caches/pip-downloads/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fpy2.py3%2Fp%2Fpip%2Fpip-1.5.5-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 1.5.4
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...

$ python -m pip --version
pip 1.5.5 from /usr/local/lib/python2.7/site-packages (python 2.7)

$ pip --version
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 2829, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 451, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 464, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python2.7/site-packages/pkg_resources.py", line 639, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pip==1.5.4

$ ls -l /usr/local/bin/pip
lrwxr-xr-x  1 robinho  admin  32 May  7 14:22 /usr/local/bin/pip -> ../Cellar/python/2.7.6_1/bin/pip

$ cat /usr/local/bin/pip
#!/usr/local/opt/python/bin/python2.7
# EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.5.4','console_scripts','pip'
__requires__ = 'pip==1.5.4'
import sys
from pkg_resources import load_entry_point

if __name__ == '__main__':
    sys.exit(
        load_entry_point('pip==1.5.4', 'console_scripts', 'pip')()
    )

@wem3
Copy link

wem3 commented May 8, 2014

I think Upgrade setuptools and pip Pull Request #29039 probably solves this issue.

A similar pip 1.5.4 to 1.5.5 problem is at the root of /homebrew/python issue #26900

@robsonpeixoto
Copy link
Author

@wem3 The Pull Request Homebrew/legacy-homebrew#29039 was merged.
Now the pip 1.5.5 are working.

Thanks for the attention, guys!

@piotr-dobrogost
Copy link

I see pkg_resources.DistributionNotFound: error popping out at SO from time to time. My diagnosis at one point in the past was

The root of the problem are often outdated scripts in the bin (Linux) or Scripts (Windows) subdirectory.

However I'm not sure what causes these scripts to not be updated when upgrading given distribution/package.

@mistydemeo
Copy link

To provide a bit more information, Homebrew's version of Python currently bundles setuptools 3.6; before yesterday it provided setuptools 3.4.1.

I can confirm that setuptools 3.6 does not fix this problem, and it does occur even if no other setuptools/distutils installations are present.

I've noticed this only happens when pip tries to upgrade itself by downloading a wheel (which is the default); pip install --upgrade --no-use-wheel pip does produce a working pip.

I can provide more information if needed.

@Ivoz
Copy link
Contributor

Ivoz commented May 8, 2014

I can also see two separate site-packages in the logs shown on the issue, which suggests to me that some migration hasn't worked properly (some scripts might have been left behind?). You only want one site-packages for one python..

@mistydemeo pip provides a RECORD file in the egg-info / dist-info folder of a package when its installed, in the python's site-packages folder. e.g

/usr/lib/python2.7/site-packages/setuptools-3.5.1-py2.7.egg-info/RECORD

It lists the files installed by pip for a package. If you compare those between pip install between --no-use-wheel and not, we could find some of the problem.

@mistydemeo
Copy link

The contents are identical between the working and non-working (non-wheel and wheel) installs; even the checksums of the files are identical.

@Ivoz
Copy link
Contributor

Ivoz commented May 8, 2014

Well, damn. @mistydemeo thanks for trying that! fwiw the reason to manually look up the RECORDs is that listing files installed with pip show --list <pkg> doesn't work for wheels in 1.5.x, it's fixed in 1.6.

I'd still consider a possibility of @robsonpeixoto is that his installation seems to have gathered two site-packages (two python install locations? :S),

/usr/local/lib/python2.7/site-packages/

and

/usr/local/Cellar/python/2.7.6_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/

I'm not sure if this is caused from homebrew workings or what. Leftover files in there might also be a problem. Having multiple setuptools (and pkg_resources.py) in these and 'wrong one' / old one being found in different situations has happened to people before.

It seems weird to me that even though python -m pip came from the correct site-packages, it installed (or not?) a script that is symlinked to an old one :S

@mistydemeo
Copy link

One of the two site-packages directories is actually just a symlink: they're the same directory. Homebrew's python is only configured to use one site-packages.

@mistydemeo
Copy link

It seems weird to me that even though python -m pip came from the correct site-packages, it installed (or not?) a script that is symlinked to an old one :S

To explain the symlink structure:

Homebrew installs software to private versioned directories within /usr/local/Cellar, then symlinked to the standard locations. In the case of Python 2.7.6 (revision 1), it is installed to /usr/local/Cellar/python/2.7.6_1, then symlinked into /usr/local. pip is not actually installed as a symlink; rather /usr/local/bin/pip just happens to be the public symlink to the real script in /usr/local/Cellar/python/2.7.6_1/bin/pip. When the working non-wheel pip upgrade occurs, it's updating the script in the longer private path, while the non-working upgrade leaves that old script in place. Neither upgrade touches the /usr/local/bin/pip symlink.

@methane
Copy link
Contributor

methane commented May 28, 2014

Homebrew download pip tarball and execute setup.py.
pip install --upgrade pip installs pip from wheel.

Installing from setup.py makes versioned script.
I think upgrading pip from wheel doesn't overwrite versioned script.
This is why DistributionNotFound happens.

@methane
Copy link
Contributor

methane commented May 29, 2014

For more detail:

$ rm -rf /usr/local/lib/python2.7/
$ brew uninstall python
Uninstalling /usr/local/Cellar/python/2.7.6_1...

$ brew install python
==> Downloading https://downloads.sf.net/project/machomebrew/Bottles/python-2.7.6_1.mavericks.bottle.tar.gz
Already downloaded: /Library/Caches/Homebrew/python-2.7.6_1.mavericks.bottle.tar.gz
==> Pouring python-2.7.6_1.mavericks.bottle.tar.gz
==> Caveats
Setuptools and Pip have been installed. To update them
  pip install --upgrade setuptools
  pip install --upgrade pip

You can install Python packages with
  pip install <package>

They will install into the site-package directory
  /usr/local/lib/python2.7/site-packages

See: https://github.com/Homebrew/homebrew/wiki/Homebrew-and-Python

.app bundles were installed.
Run `brew linkapps` to symlink these to /Applications.
Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
You can try again using:
  brew link python

Possible conflicting files are:
/usr/local/bin/pip
==> /usr/local/Cellar/python/2.7.6_1/bin/python -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/2.7.6_1/bin --install-lib=/usr/local/lib/python2.7/
==> /usr/local/Cellar/python/2.7.6_1/bin/python -s setup.py --no-user-cfg install --force --verbose --install-scripts=/usr/local/Cellar/python/2.7.6_1/bin --install-lib=/usr/local/lib/python2.7/
==> Summary
🍺  /usr/local/Cellar/python/2.7.6_1: 4907 files, 80M

$ ll /usr/local/bin/pip
lrwxr-xr-x  1 inada-n  admin  32  5 29 09:18 /usr/local/bin/pip@ -> ../Cellar/python/2.7.6_1/bin/pip

$ ll /usr/local/bin/pip2
lrwxr-xr-x  1 inada-n  admin  33  5 29 09:18 /usr/local/bin/pip2@ -> ../Cellar/python/2.7.6_1/bin/pip2

$ rm /usr/local/bin/pip2

$ pip install --upgrade pip
Downloading/unpacking pip from https://pypi.python.org/packages/py2.py3/p/pip/pip-1.5.6-py2.py3-none-any.whl#md5=4d4fb4b69df6731c7aeaadd6300bc1f2
  Using download cache from /Users/inada-n/.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fpy2.py3%2Fp%2Fpip%2Fpip-1.5.6-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 1.5.4
    Uninstalling pip:
      Successfully uninstalled pip
Successfully installed pip
Cleaning up...

$ ll /usr/local/bin/pip
lrwxr-xr-x  1 inada-n  admin  32  5 29 09:18 /usr/local/bin/pip@ -> ../Cellar/python/2.7.6_1/bin/pip

$ ll /usr/local/bin/pip2
-rwxr-xr-x  1 inada-n  admin  305  5 29 09:19 /usr/local/bin/pip2*

$ pip
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 2749, in <module>
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 446, in _build_master
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 459, in _build_from_requirements
  File "build/bdist.macosx-10.9-x86_64/egg/pkg_resources.py", line 628, in resolve
pkg_resources.DistributionNotFound: pip==1.5.4

$ pip2

Usage:
  pip <command> [options]
...

@methane
Copy link
Contributor

methane commented May 29, 2014

When using --no-use-wheel, pip overwrites to symlinked script.

$ pip install -U --no-use-wheel pip --log ~/pip.log
Downloading/unpacking pip from https://pypi.python.org/packages/source/p/pip/pip-1.5.6.tar.gz#md5=01026f87978932060cc86c1dc527903e
  Downloading pip-1.5.6.tar.gz (938kB): 938kB downloaded
  Storing download in cache at ./.pip/cache/https%3A%2F%2Fpypi.python.org%2Fpackages%2Fsource%2Fp%2Fpip%2Fpip-1.5.6.tar.gz
  Running setup.py (path:/private/var/folders/11/3pzjxxcd7q53dbfsm95ms3d00000gn/T/pip_build_inada-n/pip/setup.py) egg_info for package pip

    warning: no files found matching 'pip/cacert.pem'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no previously-included files matching '*.rst' found under directory 'docs/_build'
    no previously-included directories found matching 'docs/_build/_sources'
Installing collected packages: pip
  Found existing installation: pip 1.5.4
    Uninstalling pip:
      Successfully uninstalled pip
  Running setup.py install for pip

    warning: no files found matching 'pip/cacert.pem'
    warning: no files found matching '*.html' under directory 'docs'
    warning: no previously-included files matching '*.rst' found under directory 'docs/_build'
    no previously-included directories found matching 'docs/_build/_sources'
    Installing pip script to /usr/local/bin
    Installing pip2.7 script to /usr/local/bin
    Installing pip2 script to /usr/local/bin
Successfully installed pip
Cleaning up...

$ ll /usr/local/bin/pip
lrwxr-xr-x  1 inada-n  admin  32  5 29 09:22 /usr/local/bin/pip@ -> ../Cellar/python/2.7.6_1/bin/pip

@methane
Copy link
Contributor

methane commented Jun 6, 2014

What is the correct behavior when file exists while installing script?

  1. Overwrite (follow symlink, current hehavior of installing from source)
  2. Overwrite (replace symlink)
  3. Do nothing (Current behavior of installing from wheel)
  4. Report error

FYI, pip --exists-action=b install -U pip doesn't fix this problem.

@fake-name
Copy link

I just encountered this issue on Ubuntu 12.04 LTS with python3.4, when attempting to upgrade pip3. It appears it is most definitely not fixed.

I worked around the issue by manually editing /usr/local/bin/pip3 and changing all instances of 1.5.4 to 1.5.6, but still...

@methane
Copy link
Contributor

methane commented Jul 14, 2014

12.04? Not 14.04?

@fake-name
Copy link

durr@mainnas:/etc$ cat lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=12.04
DISTRIB_CODENAME=precise
DISTRIB_DESCRIPTION="Ubuntu 12.04.4 LTS"

Yep.

See https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes

FWIW, I'm not using any packages through the official Ubuntu repos (everything was installed with pip3, which I installed using the get-pip script.

@methane
Copy link
Contributor

methane commented Jul 14, 2014

OK, I see.

Homebrew and deadsnakes may cause same problem with different ways.
Is your /usr/loal/bin/pip3 symlink?

The bugfix for homebrew is merged but not released yet.
You may or may not be able to upgrade pip from 1.6.0 to 1.6.1.
Before that, use --no-use-wheel option when upgrading pip or setuptools.

@fake-name
Copy link

/usr/local/bin/pip3 is not a symlink (and neither is /usr/local/bin/pip3.4, actually). They're both text scripts.

Isn't the wheel group a mac-os specific thing? There is no wheel group on my computer.

@Ivoz
Copy link
Contributor

Ivoz commented Jul 14, 2014

Isn't the wheel group a mac-os specific thing?

Not really - https://en.wikipedia.org/wiki/Wheel_(Unix_term)

IIRC it's most often seen in debian, where sudo users belong to it.

In this case though, --no-use-wheel refers to the wheel packaging format.

@methane
Copy link
Contributor

methane commented Jul 14, 2014

@fake-name OK, It's a different problem from Homebrew.
Please create new issue for that.

@noodlebreak
Copy link

@Ivoz Thank you very much!! Your instructions totally fixed my broken pip.

@gzur
Copy link

gzur commented Feb 4, 2016

I just had this problem on Ubuntu 14.04 LTS after upgrading pip through pip.

Looked like this:

user@ubuntu_14_04_LTS:~$ pip
Traceback (most recent call last):
  File "/usr/local/bin/pip", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 2749, in <module>
    working_set = WorkingSet._build_master()
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 444, in _build_master
    ws.require(__requires__)
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 725, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/lib/python2.7/dist-packages/pkg_resources.py", line 628, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pip==1.5.6

After trying uninstalling both via pip and apt-get without ACTUALLY getting rid off it, I tracked down a file called pip (through which pip) in /usr/local/bin which contained the following script:

    user@ubuntu_14_04_LTS:/usr/local/bin$ cat pip
    #!/usr/bin/python
    # EASY-INSTALL-ENTRY-SCRIPT: 'pip==1.5.6','console_scripts','pip'
    __requires__ = 'pip==1.5.6'
    import sys
    from pkg_resources import load_entry_point

    if __name__ == '__main__':
        sys.exit(
            load_entry_point('pip==1.5.6', 'console_scripts', 'pip')()
        )

I just deleted this file and reinstalled with apt-get.
There were actually two more files (pip2 and pip2.7) containing similar code which I deleted for good measure.

Works now.

@lengerfulluse
Copy link

@gzur thanks, it works for me!

@lock lock bot added the auto-locked Outdated issues that have been locked by automation label Jun 2, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jun 2, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
auto-locked Outdated issues that have been locked by automation C: wheel The wheel format and 'pip wheel' command
Projects
None yet
Development

No branches or pull requests