New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installing wheels does not respect distutils.cfg #1641

Closed
schlamar opened this Issue Mar 13, 2014 · 9 comments

Comments

Projects
None yet
4 participants
@schlamar
Contributor

schlamar commented Mar 13, 2014

I've configured distutils in distutils.cfg to install to ~/lib/python2.7. It looks like this is failing with wheel installs:

[vagrant@localhost ~]$ pip-2.7 install virtualenv
Downloading/unpacking virtualenv
  Downloading virtualenv-1.11.4-py2.py3-none-any.whl (1.7MB): 1.7MB downloaded
Installing collected packages: virtualenv
Cleaning up...
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/site-packages/pip/commands/install.py", line 283, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/local/lib/python2.7/site-packages/pip/req.py", line 1435, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/pip/req.py", line 671, in install
    self.move_wheel_files(self.source_dir, root=root)
  File "/usr/local/lib/python2.7/site-packages/pip/req.py", line 901, in move_wheel_files
    pycompile=self.pycompile,
  File "/usr/local/lib/python2.7/site-packages/pip/wheel.py", line 207, in move_wheel_files
    clobber(source, lib_dir, True)
  File "/usr/local/lib/python2.7/site-packages/pip/wheel.py", line 194, in clobber
    os.makedirs(destsubdir)
  File "/usr/local/lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/usr/local/lib/python2.7/site-packages/virtualenv-1.11.4.dist-info'

Storing debug log for failure in /home/vagrant/.pip/pip.log
[vagrant@localhost ~]$ pip-2.7 install --no-use-wheel virtualenv
Downloading/unpacking virtualenv
  Downloading virtualenv-1.11.4.tar.gz (1.8MB): 1.8MB downloaded
  Running setup.py (path:/tmp/pip_build_vagrant/virtualenv/setup.py) egg_info for package virtualenv

    warning: no previously-included files matching '*' found under directory 'docs/_templates'
    warning: no previously-included files matching '*' found under directory 'docs/_build'
Installing collected packages: virtualenv
  Running setup.py install for virtualenv

    warning: no previously-included files matching '*' found under directory 'docs/_templates'
    warning: no previously-included files matching '*' found under directory 'docs/_build'
    Installing virtualenv script to /home/vagrant/bin
    Installing virtualenv-2.7 script to /home/vagrant/bin
Successfully installed virtualenv
Cleaning up...
[vagrant@localhost ~]$ 
@dstufft

This comment has been minimized.

Show comment
Hide comment
@dstufft

dstufft Mar 13, 2014

Member

What version of pip?

Member

dstufft commented Mar 13, 2014

What version of pip?

@schlamar

This comment has been minimized.

Show comment
Hide comment
@schlamar

schlamar Mar 13, 2014

Contributor
$ pip-2.7 --version
pip 1.5.4 from /usr/local/lib/python2.7/site-packages (python 2.7)
Contributor

schlamar commented Mar 13, 2014

$ pip-2.7 --version
pip 1.5.4 from /usr/local/lib/python2.7/site-packages (python 2.7)
@qwcode

This comment has been minimized.

Show comment
Hide comment
@qwcode

qwcode Mar 13, 2014

Contributor

can you paste your distutils.cfg? and what it's location?

#1297 (in pip 1.5) made wheel aware of distutils.cfg files, and at the time, we had verification that it was working (at least for the specific scenario we were looking at on OSX)

Contributor

qwcode commented Mar 13, 2014

can you paste your distutils.cfg? and what it's location?

#1297 (in pip 1.5) made wheel aware of distutils.cfg files, and at the time, we had verification that it was working (at least for the specific scenario we were looking at on OSX)

@schlamar

This comment has been minimized.

Show comment
Hide comment
@schlamar

schlamar Mar 13, 2014

Contributor

Ah, I could fix this issue by defining install_platlib and install_purelib in my distutils.cfg.

[install]
install_lib = ~/lib/python2.7
install_platlib = ~/lib/python2.7
install_purelib = ~/lib/python2.7
install_scripts = ~/bin

But this is just a workaround. Setting install_lib should take precedence over platlib and purelib (see http://hg.python.org/cpython/file/b44a3f6676b8/Lib/distutils/command/install.py#l342). It defaults to None:

>>> d = Distribution({'name': 1})
>>> d.parse_config_files()
>>> i = d.get_command_obj('install', create=True)
>>> repr(i.install_lib)
'None'
Contributor

schlamar commented Mar 13, 2014

Ah, I could fix this issue by defining install_platlib and install_purelib in my distutils.cfg.

[install]
install_lib = ~/lib/python2.7
install_platlib = ~/lib/python2.7
install_purelib = ~/lib/python2.7
install_scripts = ~/bin

But this is just a workaround. Setting install_lib should take precedence over platlib and purelib (see http://hg.python.org/cpython/file/b44a3f6676b8/Lib/distutils/command/install.py#l342). It defaults to None:

>>> d = Distribution({'name': 1})
>>> d.parse_config_files()
>>> i = d.get_command_obj('install', create=True)
>>> repr(i.install_lib)
'None'
@schlamar

This comment has been minimized.

Show comment
Hide comment
@schlamar

schlamar Mar 13, 2014

Contributor

I'll submit a PR shortly.

Contributor

schlamar commented Mar 13, 2014

I'll submit a PR shortly.

@piotr-dobrogost

This comment has been minimized.

Show comment
Hide comment
@piotr-dobrogost

piotr-dobrogost Mar 13, 2014

I see you're installing into home directory. In that case you might consider user scheme.
From http://docs.python.org/dev/distutils/install.html#alternate-installation-the-user-scheme

This scheme is designed to be the most convenient solution for users that don’t have write permission to the global site-packages directory or don’t want to install into it.

Just in case you didn't know which I guess is not the case :)

piotr-dobrogost commented Mar 13, 2014

I see you're installing into home directory. In that case you might consider user scheme.
From http://docs.python.org/dev/distutils/install.html#alternate-installation-the-user-scheme

This scheme is designed to be the most convenient solution for users that don’t have write permission to the global site-packages directory or don’t want to install into it.

Just in case you didn't know which I guess is not the case :)

@schlamar

This comment has been minimized.

Show comment
Hide comment
@schlamar

schlamar Mar 13, 2014

Contributor

Just in case you didn't know which I guess is not the case :)

I know about that but thanks =)

Contributor

schlamar commented Mar 13, 2014

Just in case you didn't know which I guess is not the case :)

I know about that but thanks =)

@piotr-dobrogost

This comment has been minimized.

Show comment
Hide comment
@piotr-dobrogost

piotr-dobrogost Mar 13, 2014

In the first moment the fact that more specific settings are overridden by less specific seemed to be illogical to me. However these are special kind of settings as these are all overrides so if we override something more general there's no sense in overriding something specific I guess.
From http://docs.python.org/dev/distutils/install.html#custom-installation :

using --install-lib will override values computed or given for --install-purelib and --install-platlib, and is recommended for schemes that don’t make a difference between Python and extension modules.

Just thinking out loud :)

piotr-dobrogost commented Mar 13, 2014

In the first moment the fact that more specific settings are overridden by less specific seemed to be illogical to me. However these are special kind of settings as these are all overrides so if we override something more general there's no sense in overriding something specific I guess.
From http://docs.python.org/dev/distutils/install.html#custom-installation :

using --install-lib will override values computed or given for --install-purelib and --install-platlib, and is recommended for schemes that don’t make a difference between Python and extension modules.

Just thinking out loud :)

@schlamar

This comment has been minimized.

Show comment
Hide comment
@schlamar

schlamar Mar 14, 2014

Contributor

I guess the 90% case is that you don't distinguish between purelib and platlib, so having a single setting to override both of them is a good thing IMO.

However, the main issue is that pip was inconsistent when installing wheels vs installing with --no-use-wheel which should be fixed now.

Contributor

schlamar commented Mar 14, 2014

I guess the 90% case is that you don't distinguish between purelib and platlib, so having a single setting to override both of them is a good thing IMO.

However, the main issue is that pip was inconsistent when installing wheels vs installing with --no-use-wheel which should be fixed now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment