Sudo usage breaks pip. #982

Closed
harlowja opened this Issue Jun 6, 2013 · 15 comments

Projects

None yet

4 participants

@harlowja
harlowja commented Jun 6, 2013

Any usage of sudo + pip (not in a venv) seems to break pip due to the new temporary directory check & mkdir creation. A simple easily reproducible example that includes one command and a second command that will break due to the new directory uid check (which doesn't play nice with sudo).

It would seem that --help (and likely other commands) should work for all users, and not require write access to some temporary folder for building (when --help is not a build command).

$ pip-python --version
pip 1.3.1 from /usr/lib/python2.6/site-packages (python 2.6)
$ sudo pip-python --help
The temporary folder for building (/tmp/pip-build-harlowja) is not owned by your user!
pip will not work until the temporary folder is either deleted or owned by your user account.
Traceback (most recent call last):
  File "/usr/bin/pip-python", line 9, in <module>
    load_entry_point('pip==1.3.1', 'console_scripts', 'pip')()
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 343, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2354, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2060, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/lib/python2.6/site-packages/pip/__init__.py", line 9, in <module>
    from pip.util import get_installed_distributions, get_prog
  File "/usr/lib/python2.6/site-packages/pip/util.py", line 15, in <module>
    from pip.locations import site_packages, running_under_virtualenv, virtualenv_no_global
  File "/usr/lib/python2.6/site-packages/pip/locations.py", line 64, in <module>
    build_prefix = _get_build_prefix()
  File "/usr/lib/python2.6/site-packages/pip/locations.py", line 54, in _get_build_prefix
    raise pip.exceptions.InstallationError(msg)
pip.exceptions.InstallationError: The temporary folder for building (/tmp/pip-build-harlowja) is not owned by your user!

It also works the other way around (which is likely even worse since now any pip command the user will try to do will not work from here on out due to the user not being able to write a directory that is root owned).

$ sudo pip-python --version
pip 1.3.1 from /usr/lib/python2.6/site-packages (python 2.6)
$ pip-python --help
The temporary folder for building (/tmp/pip-build-harlowja) is not owned by your user!
pip will not work until the temporary folder is either deleted or owned by your user account.
Traceback (most recent call last):
  File "/usr/bin/pip-python", line 9, in <module>
    load_entry_point('pip==1.3.1', 'console_scripts', 'pip')()
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 343, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2354, in load_entry_point
    return ep.load()
  File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2060, in load
    entry = __import__(self.module_name, globals(),globals(), ['__name__'])
  File "/usr/lib/python2.6/site-packages/pip/__init__.py", line 9, in <module>
    from pip.util import get_installed_distributions, get_prog
  File "/usr/lib/python2.6/site-packages/pip/util.py", line 15, in <module>
    from pip.locations import site_packages, running_under_virtualenv, virtualenv_no_global
  File "/usr/lib/python2.6/site-packages/pip/locations.py", line 64, in <module>
    build_prefix = _get_build_prefix()
  File "/usr/lib/python2.6/site-packages/pip/locations.py", line 54, in _get_build_prefix
    raise pip.exceptions.InstallationError(msg)
pip.exceptions.InstallationError: The temporary folder for building (/tmp/pip-build-harlowja) is not owned by your user!
@qwcode
Contributor
qwcode commented Jun 7, 2013

2 side points:

  1. the fact that the build dir is created/confirmed on import (vs when it's actually needed) was brought up during the initial review, but it ended up being merged this way.
  2. these fixed build dirs are likely to go away with #906 anyway.
@harlowja
harlowja commented Jun 7, 2013

Thanks, very good to know its going to be fixed.

@qwcode
Contributor
qwcode commented Jun 8, 2013

cc @d1b

this problem is not universal. I don't have this issue on ubuntu or centos.
what distro are you on?

can you give me the results from these:

sudo python -c "import getpass; print(getpass.getuser())"
sudo python -c "import os; print(os.environ.get('LOGNAME'))"
sudo python -c "import pwd, os; print(pwd.getpwuid(os.geteuid()).pw_name)"

@harlowja
harlowja commented Jun 8, 2013

This is RHEL6.2

-bash-4.1$ sudo python -c "import getpass; print(getpass.getuser())"
harlowja
-bash-4.1$ sudo python -c "import os; print(os.environ.get('LOGNAME'))"
harlowja
-bash-4.1$ sudo python -c "import pwd, os; print(pwd.getpwuid(os.geteuid()).pw_name)"
root

@qwcode
Contributor
qwcode commented Jun 8, 2013

thanks @harlowja . all my answers were 'root'.
I guess for unix, we can use pwd.getpwuid(os.geteuid()).pw_name.
for windows, it has to be getpass.getuser(), since geteuid only works on unix.
I'll post a pull later and have you confirm it works.

@harlowja
harlowja commented Jun 8, 2013

Ya, thats pretty weird. I wonder if since RHEL6.2 uses 2.6 that 2.6 has this behavior? But then centos should also. Pretty odd...

@d1b
Contributor
d1b commented Jun 11, 2013

@qwcode this is very interesting indeed. On ubuntu 13.04 with python 2.7.4,
sudo python -c "import getpass;print(getpass.getuser())"

prints out 'root'. It would seem that 'sudo' on some systems end up with environmental variables that differ from the norm. So we should change it to bepwd.getpwuid(os.geteuid()).pw_name on unix. In addition, we will need to change the if file_uid != os.getuid(): check.

@russkel
russkel commented Jun 11, 2013

Same issue for me on OSX. Build dirs are created as root if running via sudo. I was going to write an issue but this seems to cover it. In this case I was using virtualenv and virtualenvwrapper, I accidently used sudo pip install markdown, Ctrl+c'ed quickly, tried the command without sudo and sure enough I couldn't write to the build dir.

@d1b
Contributor
d1b commented Jun 11, 2013

@russkel what's the output you see (from pip) ?

@russkel
russkel commented Jun 11, 2013

@d1b this is what it chucked out. I rm'ed the dir as sudo and I am on my merry way.

(project)Samorost:project russ$ pip install markdown
Downloading/unpacking markdown
  Downloading Markdown-2.3.1.tar.gz (267kB): 267kB downloaded
Exception:
Traceback (most recent call last):
  File "/Users/russ/.virtualenvs/project/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/Users/russ/.virtualenvs/project/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 266, in run
    requirement_set.prepare_files(finder, force_root_egg_info=self.bundle, bundle=self.bundle)
  File "/Users/russ/.virtualenvs/project/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/req.py", line 1033, in prepare_files
    self.unpack_url(url, location, self.is_download)
  File "/Users/russ/.virtualenvs/project/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/req.py", line 1161, in unpack_url
    retval = unpack_http_url(link, location, self.download_cache, self.download_dir)
  File "/Users/russ/.virtualenvs/project/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/download.py", line 559, in unpack_http_url
    unpack_file(temp_location, location, content_type, link)
  File "/Users/russ/.virtualenvs/project/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/util.py", line 590, in unpack_file
    untar_file(filename, location)
  File "/Users/russ/.virtualenvs/project/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/util.py", line 504, in untar_file
    os.makedirs(location)
  File "/Users/russ/.virtualenvs/project/bin/../lib/python2.7/os.py", line 157, in makedirs
    mkdir(name, mode)
OSError: [Errno 13] Permission denied: '/Users/russ/.virtualenvs/project/build/markdown'
@russkel
russkel commented Jun 11, 2013

From my shell backlog:
drwxr-xr-x 3 root staff 102 11 Jun 18:57 build

@d1b
Contributor
d1b commented Jun 11, 2013

@russkel the issue you hit is not related to this issue (afaik).

@qwcode
Contributor
qwcode commented Jun 13, 2013

@d1b I started to make the change to use pwd.getpwuid(os.geteuid()).pw_name on unix, but ran into needing to fix the mocking in your tests, and didn't have the time. feel free to submit a pull, with the test fixes, and it will be a quick merge.

@d1b
Contributor
d1b commented Jun 15, 2013

@qwcode sure thing.

@qwcode qwcode pushed a commit that closed this issue Jun 15, 2013
@d1b d1b Fix #982 by using the effective user id on unix systems instead of de…
…pending on

getpass.getuser(). This is required because on some systems 'LOGNAME'
is not updated by sudo.
Signed-off-by: david <db@d1b.org>
267c79d
@qwcode qwcode closed this in 267c79d Jun 15, 2013
@harlowja harlowja added a commit to harlowja/packtools that referenced this issue Jan 4, 2014
@harlowja Joshua Harlow + harlowja Cleanups after new changes/features.
- Some style adjustments.
- Use SRC_REPOS for mapping repo name to source repository name.
- Split the build_binary into individual tiny functions.
- Fix up the progress bars and iterable logging.
- Ensure that we log that we wrote to /etc/yum.repos.d and leave a trace for later cleanup.
- Quiet some of the new executes (and put there output to files).
- Add a smithy clean_pip() function that will help avoid hitting pypa/pip#982
- Further cleanup of smithy after it being partially rewritten.
- Instead of hard coding 'conf/distros/rhel.yaml' in smithy, take this from the sourced file.
- Ensure that we can't remove packages smithy requires to operate.
- Ensure the package version that py2rpm is building is also trimmed of zeros.

Change-Id: I2df8a47f0115de2684777b64db42e08d50ef4115
4928c9f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment