pip fails to install projects with console entry points on Python 3.6

[pfmoore]

• Pip version: 8.1.2
• Python version: 3.6b0
• Operating System: Windows 7

>py -m pip --version
pip 8.1.2 from C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages (python 3.6)

Description:

This appears to happen for any package with console entry points, but not for packages without. It looks like a distlib bug, so it will need fixing there and revendoring.

Further note, this issue seems to affect python -m pip install -U pip so we'll need to consider how to help users to upgrade past this issue. (Simplest way may be to recommend that users uninstall pip and then use get-pip.py).

What I've run:

>py -m pip install invoke
Collecting invoke
  Using cached invoke-0.13.0-py3-none-any.whl
Installing collected packages: invoke
Exception:
Traceback (most recent call last):
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\commands\install.py", line 317, in run
    prefix=options.prefix_path,
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\req\req_set.py", line 742, in install
    **kwargs
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\req\req_install.py", line 831, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\req\req_install.py", line 1032, in move_wheel_files
    isolated=self.isolated,
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\wheel.py", line 493, in move_wheel_files
    maker.make_multiple(['%s = %s' % kv for kv in console.items()])
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 383, in make_multiple
    filenames.extend(self.make(specification, options))
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 372, in make
    self._make_script(entry, filenames, options=options)
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 276, in _make_script
    self._write_script(scriptnames, shebang, script, filenames, ext)
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 212, in _write_script
    launcher = self._get_launcher('t')
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 351, in _get_launcher
    result = finder(distlib_package).find(name).bytes
  File "C:\Users\UK03306\AppData\Local\Programs\Python\Python36\lib\site-packages\pip\_vendor\distlib\resources.py", line 324, in finder
    raise DistlibException('Unable to locate finder for %r' % package)
pip._vendor.distlib.DistlibException: Unable to locate finder for 'pip._vendor.distlib'

Presumably this is distlib issue https://bitbucket.org/pypa/distlib/issues/87/resource-finder-looking-at-wrong-module - is that correct @zooba?

[pfmoore]

Just to be clear, assuming this is a distlib issue, there's no immediate action needed for pip. I raised this issue mainly as a reminder that we need to get a fix out relatively quickly once the distlib issue is resolved, as the problem makes it particularly hard for users to test the new Python 3.6 release.

[zooba]

Yep, that's the issue. I was just coming over here to link to it :)

[zooba]

Do you know of any env var or CLI option to skip entry points? I couldn't find anything quickly scanning the docs and code

[pfmoore]

Not offhand, unfortunately.

[zooba]

Looks like Vinay just resolved the issue. It would be nice to get pip updated too.

[pfmoore]

We'll need @vsajip to release a new version of distlib, which we can then vendor. I'm happy to do the re-vendor but I can't produce a pip release. @dstufft would we be able to do a quick pip release for this?

Without an easy way to disable scripts, allowing an upgrade, maybe this would be worth a fixed Python 3.6b1 Windows binary with the updated pip in it? Unfortunately, that wouldn't just be the distlib fix, so it may not be a good idea to have pip different in the Linux and Windows builds.

[stonebig]

Hi. I get a " 'pip' has no attribute 'get installed distributions'" (on3.6.0b1-amd64 win10). is it same sort of issue or a new one ?

found the/my error : my install process was doing a "python -m pip install --upgrade --force-reinstall pip" that ends badly when pip version is unchanged. Sorry for the noise

... patching resources.py doesn't seem to be enough tp resolve the issue, for Winpython at least, as I get now the same initial error as pfmoore.

[zooba]

Guido unequivocally stated last week that quick fixes like that should up the version number, and I'm inclined to agree. I think uninstalling and running get-pip.py is not unreasonable.

[zooba]

@stonebig I think your install is corrupt - have you tried repairing it?

[pfmoore]

@stonebig Doesn't sound like it, but could you give more details (how you invoked pip, etc). I presume this is with 3.6b1 - is it with the pip shipped with that or with master?

@zooba Sounds reasonable, like you say a one-off get-pip isn't that onerous for people running the beta.

[stonebig]

Python 3.6.0b1 (64-bit)_20160913185823.log.txt
the message is on the "zipped" result. for what is in the "installed programs", it propose me to install it when I click on "uninstall" (only option), and I get the attached log

idle works, on the winpython3.6 beta directory, so things are not an absolute fail.

[zooba]

@pfmoore Except I did just try py get-pip.py and it tried to install into the wrong Python (i.e. a different one than py runs with). Probably a launcher bug...

[zooba]

@stonebig I think we need you to write up the step-by-step list of everything you did and post it on https://bugs.python.org/. I'm fairly convinced your issues are not bugs in CPython or pip, but they may be an installer issue. Without specific steps and the download URLs you used, I can't make any better guesses.

[pfmoore]

@zooba get-pip.py has a shebang of #!/usr/bin/env python. That's going to run whatever Python command you have on your PATH by default. Did you perchance not have Python 3.6 on PATH at the time?

[stonebig]

https://bugs.python.org/issue28132

1- I did the zooba style of install of python3.6, to get a zippable thing,
2- after the zip creation for winpython , i went to the "Applications and functionnality" panel of windows 10 and tried to uninstall. the uninstall button showed me the install door only ... about 5 times out of 6.

I'm windows10-10586.545 (not upgraded to summer 2016 yet)

[stonebig]

ok, by clicking on the ".exe" installer, I succeeded to repair, then via "application and functionalities menu" uninstall started the right way, finished happy of itself ... without uninstalling from "applications and functionalities".

clicking again on the installer (instead of application and functionnalities) did at last the job of uninstalling

[coreyrshaw]

http://stackoverflow.com/questions/39908406/unable-to-locate-finder-for-pip-vendor-distlib-error-when-using-pip-instal

[coreyrshaw]

Running into this issue :(

[pfmoore]

I assume the fix is in the recently released distlib 0.2.4, so we should revendor. (Probably better if someone on Unix does this, IIRC git gets confused over line endings if I do it on Windows :-()

[zooba]

Yes, it's in 0.2.4 (listed under "Resources" at https://goo.gl/M3kQzR)

[brettcannon]

I just tried to update to pip 9.0.0 and I'm still seeing the error under Python 3.6.0b3:

Collecting pip
  Using cached pip-9.0.0-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 8.1.2
    Uninstalling pip-8.1.2:
      Successfully uninstalled pip-8.1.2
  Rolling back uninstall of pip
Exception:
Traceback (most recent call last):
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\basecommand.py", line 215, in main
    status = self.run(options, args)
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\commands\install.py", line 317, in run
    prefix=options.prefix_path,
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\req\req_set.py", line 742, in install
    **kwargs
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\req\req_install.py", line 831, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\req\req_install.py", line 1032, in move_wheel_files
    isolated=self.isolated,
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\wheel.py", line 463, in move_wheel_files
    generated.extend(maker.make(spec))
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 372, in make
    self._make_script(entry, filenames, options=options)
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 276, in _make_script
    self._write_script(scriptnames, shebang, script, filenames, ext)
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 212, in _write_script
    launcher = self._get_launcher('t')
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\_vendor\distlib\scripts.py", line 351, in _get_launcher
    result = finder(distlib_package).find(name).bytes
  File "C:\Users\brcan\Desktop\venv-py36\lib\site-packages\pip\_vendor\distlib\resources.py", line 324, in finder
    raise DistlibException('Unable to locate finder for %r' % package)
pip._vendor.distlib.DistlibException: Unable to locate finder for 'pip._vendor.distlib'

[stonebig]

maybe you need to wait python-3.6.0b4, to have pip-9.0.1 bundled inside ?

[zooba]

@brettcannon Yeah, we didn't get the newer version into b3, and the error comes from the installed version, not the one being installed. (However, uninstall followed by get-pip.py should be fine now, I'd imagine.)

[ghostsquad]

this appears to still be a problem for 3.6.0 for Windows (release date: 2016-12-23)

(wordsearch_solver) C:\Users\ghost\python_virtualenvs\wordsearch_solver\Scripts>.\python.exe -m pip install --upgrade pip
You are using pip version 7.1.0, however version 9.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Collecting pip
  Using cached pip-9.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 7.1.0
    Uninstalling pip-7.1.0:
      Successfully uninstalled pip-7.1.0
  Rolling back uninstall of pip
Exception:
Traceback (most recent call last):
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\basecommand.py", line 223, in main
    status = self.run(options, args)
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\commands\install.py", line 299, in run
    root=options.root_path,
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\req\req_set.py", line 646, in install
    **kwargs
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\req\req_install.py", line 813, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\req\req_install.py", line 1008, in move_wheel_files
    isolated=self.isolated,
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\wheel.py", line 449, in move_wheel_files
    generated.extend(maker.make(spec))
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\_vendor\distlib\scripts.py", line 323, in make
    self._make_script(entry, filenames, options=options)
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\_vendor\distlib\scripts.py", line 227, in _make_script
    self._write_script(scriptnames, shebang, script, filenames, ext)
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\_vendor\distlib\scripts.py", line 163, in _write_script
    launcher = self._get_launcher('t')
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\_vendor\distlib\scripts.py", line 302, in _get_launcher
    result = finder(distlib_package).find(name).bytes
  File "C:\Users\ghost\python_virtualenvs\wordsearch_solver\lib\site-packages\pip\_vendor\distlib\resources.py", line 297, in finder
    raise DistlibException('Unable to locate finder for %r' % package)
pip._vendor.distlib.DistlibException: Unable to locate finder for 'pip._vendor.distlib'


(wordsearch_solver) C:\Users\ghost\python_virtualenvs\wordsearch_solver\Scripts>python --version
Python 3.6.0

[zooba]

@ghostsquad The bug was in pip prior to 9.0.1. Your virtual environment has pip 7.1.0, which includes the bug, and so it's nothing to do with Python 3.6.

You should either recreate your virtual environment, or delete the pip installation from site-packages completely (I've found that pip uninstall pip doesn't work reliably once you've failed to upgrade it) and use the get-pip.py script to install it again.

[sirberus]

@zooba Fresh install of Python 3.6, get-pip.py is encountering this error that was supposedly resolved in 3.5. Any advice? Right now I'm SOL for updating pip.

EDIT - I was able to resolve this with the following steps:

• placing my current pip (v8.1.2) folder in Lib rather than Lib\site-packages so that pip could properly install 9.0.1 in Lib\site-packages
• Commenting out / deleting all lines within the try clause of main() in get-pip.py except for the line that calls get-pip.bootstrap():

try:
    bootstrap(tmpdir=tmpdir)
finally:
    ...

This forced get-pip.py to use the 8.1.2 instead of its internal pip binary which was running into the zlib error I indicated in my original comment.

[opbod]

Same problem here on Windows 10: fresh Python 3.6 install, fresh venv.

(venv-3.6) C:\Users\Myuser\Documents>python -m pip install --upgrade pip
You are using pip version 7.1.0, however version 9.0.1 is available.
You should consider upgrading via the 'python -m pip install --upgrade pip' command.
Collecting pip
  Using cached pip-9.0.1-py2.py3-none-any.whl
Installing collected packages: pip
  Found existing installation: pip 7.1.0
	Uninstalling pip-7.1.0:
	  Successfully uninstalled pip-7.1.0
  Rolling back uninstall of pip
Exception:
Traceback (most recent call last):
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\basecommand.py", line 223, in main
	status = self.run(options, args)
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\commands\install.py", line 299, in run
	root=options.root_path,
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\req\req_set.py", line 646, in install
	**kwargs
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\req\req_install.py", line 813, in install
	self.move_wheel_files(self.source_dir, root=root)
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\req\req_install.py", line 1008, in move_wheel_files
	isolated=self.isolated,
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\wheel.py", line 449, in move_wheel_files
	generated.extend(maker.make(spec))
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\_vendor\distlib\scripts.py", line 323, in make
	self._make_script(entry, filenames, options=options)
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\_vendor\distlib\scripts.py", line 227, in _make_script
	self._write_script(scriptnames, shebang, script, filenames, ext)
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\_vendor\distlib\scripts.py", line 163, in _write_script
	launcher = self._get_launcher('t')
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\_vendor\distlib\scripts.py", line 302, in _get_launcher
	result = finder(distlib_package).find(name).bytes
  File "C:\Users\Myuser\venv-3.6\lib\site-packages\pip\_vendor\distlib\resources.py", line 297, in finder
	raise DistlibException('Unable to locate finder for %r' % package)
pip._vendor.distlib.DistlibException: Unable to locate finder for 'pip._vendor.distlib'


(venv-3.6) C:\Users\Myuser\Documents>

Fixed by deleting all pip related directories under C:\Users\Myuser\venv-3.6\Lib\site-packages\, downloading get-pip.py and executing the command, as mentioned above by @zooba .

[ngse]

As above, but fixed by running easy_install pip after removing all traces of pip from my venv.

[mgoodfellow]

I quickly created a bash script to generate patched venvs on Windows using the information from above. Hope this helps someone generate their venvs a little bit faster!

I chucked it in a Gist

Code inline here:

#!/bin/bash

# This is a handy script to apply a pip fix to the venv created by python 3.6 on Windows
# Written by Mike Goodfellow in a few minutes - feel free to improve this!!
# Based on information in https://github.com/pypa/pip/issues/3964
#
# NOTE: Currently works if python on path points to python3 which is commonly
#       does on Windows.

if [ $# -ne 1 ]
then
	echo "Usage: venv_init /path/to/venv"
	exit 1
fi

echo "Creating new venv in $1..."

# NOTE: This needs to be on the path...
python -m venv $1

echo "Applying the pip fix..."

rm -rf $1/Lib/site-packages/pip*

$1/Scripts/easy_install pip

$1/Scripts/pip install --upgrade pip setuptools

echo "Patched version of venv has been configured in $1"

Written in 5 mins, so feel free to improve