Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Homebrew Python, Macvim: getting user scheme error (--user) #552

Closed
oryband opened this issue Jun 19, 2013 · 23 comments

Comments

Projects
None yet
@oryband
Copy link

commented Jun 19, 2013

I'm using a Homebrew Python & Macvim, on OSX ML.
When trying to pip install --user git+git://github.com/Lokaltog/powerline, I get a user scheme related error:

------------------------------------------------------------
/usr/local/bin/pip run on Wed Jun 19 03:11:30 2013
Downloading/unpacking git+git://github.com/Lokaltog/powerline

  Cloning git://github.com/Lokaltog/powerline to /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build

  Found command 'git' at '/usr/local/bin/git'
  Running command /usr/local/bin/git clone -q git://github.com/Lokaltog/powerline /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build
  Running setup.py egg_info for package from git+git://github.com/Lokaltog/powerline

    running egg_info
    creating pip-egg-info/Powerline.egg-info
    writing requirements to pip-egg-info/Powerline.egg-info/requires.txt
    writing pip-egg-info/Powerline.egg-info/PKG-INFO
    writing top-level names to pip-egg-info/Powerline.egg-info/top_level.txt
    writing dependency_links to pip-egg-info/Powerline.egg-info/dependency_links.txt
    writing manifest file 'pip-egg-info/Powerline.egg-info/SOURCES.txt'
    warning: manifest_maker: standard file '-c' not found


    reading manifest file 'pip-egg-info/Powerline.egg-info/SOURCES.txt'
    reading manifest template 'MANIFEST.in'
    writing manifest file 'pip-egg-info/Powerline.egg-info/SOURCES.txt'
  Source in /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build has version beta, which satisfies requirement Powerline==beta from git+git://github.com/Lokaltog/powerline
  skipping extra docs
Installing collected packages: Powerline

  Running setup.py install for Powerline

    Running command /usr/local/opt/python/bin/python2.7 -c "import setuptools;__file__='/var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-JsOX6p-record/install-record.txt --single-version-externally-managed --user
    running install
    error: can't combine user with prefix, exec_prefix/home, or install_(plat)base

    Complete output from command /usr/local/opt/python/bin/python2.7 -c "import setuptools;__file__='/var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-JsOX6p-record/install-record.txt --single-version-externally-managed --user:

    running install

error: can't combine user with prefix, exec_prefix/home, or install_(plat)base

----------------------------------------

Command /usr/local/opt/python/bin/python2.7 -c "import setuptools;__file__='/var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-JsOX6p-record/install-record.txt --single-version-externally-managed --user failed with error code 1 in /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build

Exception information:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
    status = self.run(options, args)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 271, in run
    requirement_set.install(install_options, global_options, root=options.root_path)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/req.py", line 1185, in install
    requirement.install(install_options, global_options, *args, **kwargs)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/req.py", line 592, in install
    cwd=self.source_dir, filter_stdout=self._filter_install, show_stdout=False)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/util.py", line 662, in call_subprocess
    % (command_desc, proc.returncode, cwd))
InstallationError: Command /usr/local/opt/python/bin/python2.7 -c "import setuptools;__file__='/var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build/setup.py';exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-JsOX6p-record/install-record.txt --single-version-externally-managed --user failed with error code 1 in /var/folders/k9/39bn572j64785jmvr63tqm180000gn/T/pip-CeX1GF-build

Any idea how to fix this?

@eddiemonge

This comment has been minimized.

Copy link

commented Jun 20, 2013

I am getting this as well now

@a-b

This comment has been minimized.

Copy link

commented Jun 22, 2013

me too

@kurtisnelson

This comment has been minimized.

Copy link

commented Jun 26, 2013

👍

@dsdeiz

This comment has been minimized.

Copy link

commented Jun 27, 2013

I got it working following this.

@oryband

This comment has been minimized.

Copy link
Author

commented Jun 27, 2013

@dsdeiz This fixes the problem.

This should be stated in the docs.

@Lokaltog

This comment has been minimized.

Copy link
Member

commented Jun 28, 2013

@oryband Feel free to add it and submit a pull request.

@a-b

This comment has been minimized.

Copy link

commented Jun 28, 2013

I wouldn't add an empty prefix as suggested at #552 (comment)

With empty prefix it will install packages to the ~/Library/Python/2.7/lib/python/site-packages/
Instead if you're using python installed by brew, you're expecting to have all your packages under /usr/local/lib/python2.7/site-packages as explained at brew info python

more info here https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python

@oryband

This comment has been minimized.

Copy link
Author

commented Jun 29, 2013

@Lokaltog What do you suggest should be done?

@unphased

This comment has been minimized.

Copy link

commented Jul 13, 2013

@a-b What should we do?

I had to do this to get it installed, it sure looks like some stuff got dumped in ~/Library...

lust@Stevens-MacBook-Pro:~ 
$ echo "[install]\nprefix=\n" > ~/.pydistutils.cfg                                              
lust@Stevens-MacBook-Pro:~ 
$ pip install --user git+git://github.com/Lokaltog/powerline
Downloading/unpacking git+git://github.com/Lokaltog/powerline
  Cloning git://github.com/Lokaltog/powerline to /var/folders/z_/ybb_x1f96l9d2yqr18ncmg1w0000gn/T/pip-
i64Eno-build
  Running setup.py egg_info for package from git+git://github.com/Lokaltog/powerline

Installing collected packages: Powerline
  Running setup.py install for Powerline

    changing mode of build/scripts-2.7/powerline from 644 to 755
    changing mode of build/scripts-2.7/powerline-lint from 644 to 755
    changing mode of /Users/lust/Library/Python/2.7/bin/powerline to 755
    changing mode of /Users/lust/Library/Python/2.7/bin/powerline-lint to 755
Successfully installed Powerline
Cleaning up...
@gji

This comment has been minimized.

Copy link

commented Jul 16, 2013

In the Homebrew documentation at https://github.com/mxcl/homebrew/wiki/Homebrew-and-Python:

Note, pip install --user is disabled for brewed Python. This is a bug in distutils, because Homebrew writes a distutils.cfg that sets the prefix for packages.

I installed powerline without the --user flag, which of course means that it is not installed in "/Users/..." but rather in "/usr/local/...." (or whatever your brew prefix is). According to the brew documentation, installing in ~/Library/Python/2.7/lib/python/site-packages/ will also work though.

@samueljohn

This comment has been minimized.

Copy link

commented Aug 12, 2013

So, if you use a brewed python (via brew install python), you can just do
pip install git+git://github.com/Lokaltog/powerline and the powerline module will be at $(brew --prefix)/lib/python2.7/site-packages. Don't need to set the PYTHONPATH.
But you have to make sure that you brew install vim after you brewed python, so that vim uses brewed python.
To check this, I use xcrun otool -L $(brew --prefix)/bin/vim and look for the linked Python.

Another option, to work around the "can't combine user with prefix, exec_prefix/home, or install_(plat)base" is to only disable the prefix temporarily:

pip install --user --install-option="--prefix=" git+git://github.com/Lokaltog/powerline

Note that this will put executable scripts into ~/Library/Python/2.7/bin.

@oryband

This comment has been minimized.

Copy link
Author

commented Aug 12, 2013

This is too complicated.

@samueljohn

This comment has been minimized.

Copy link

commented Aug 12, 2013

I am even thinking to let homebrew patch distutils such that --user overwrites the prefix.

@Lokaltog

This comment has been minimized.

Copy link
Member

commented Aug 13, 2013

Haha, this is ridiculous. I can't believe that OS X causes this many issues with installing a simple Python package. Is there any of this stuff that should be put into the docs? I can't test any of this myself so I'm relying on OS X users to provide correct installation instructions.

@samueljohn

This comment has been minimized.

Copy link

commented Aug 13, 2013

Wrong. This is no OS X issue. Not at all. Python's distutils should give --user precedence over the prefix set in a distutils.cfg file.

@Lokaltog

This comment has been minimized.

Copy link
Member

commented Aug 13, 2013

I think the general Python user/system path mess is OS X specific (i.e. different Python paths depending on if Python is native or installed with brew, and different paths depending on if vim was compiled before or after Python had been installed with brew, and issues with --user not working). I've not been able to reproduce any of these issues on Linux.

It could of course also be a bug in distutils which is triggered in this scenario where --user is not given precedence over distutils.cfg.

@samueljohn

This comment has been minimized.

Copy link

commented Aug 14, 2013

I'd not say it is a mess. On Mac OS X, Python decided (not Apple) to put user related packages into ~/Library/Python/... instead of ~/.python or some other dot-directory. A global system wide site-packages is at the root /Library/Python. I think this is not bad.

[...] different Python paths depending on if Python is native or installed with brew

Again this is not true. Both system and brewed Python respect the above mentioned paths for packages.
Only thing, however, is that system Python does not know of the site-packages brew (and the pip that comes with a brewed python) installs to: $(brew --prefix)/lib/pythonX.Y/site-packages.

This is so due to several reasons, but mostly because homebrew promises not to write files outside of the brew --prefix and also because every python has a built-in location for site-packages at the sys.prefix. A brewed Python is no exception here.

What is indeed true is that Homebrew writes a distutils.cfg specifying the prefix for packages to be installed with distutils (so for pip) to be equal to the brew --prefix. Most linux don't do this - everything goes into /usr/local (I guess).

The only unfortunate thing is that distutils does not give more weight to --user. It is usually good practice that explicitly given flags overwrite some defaults from a configuration file.

Let me stress that --user is not necessary at all for a brewed python and vim:

$ brew install python
$ brew install vim
$ pip install git+git://github.com/Lokaltog/powerline

That is perfectly fine and I run my vim with powerline this way. Btw: Thanks for creating this software! 🍻

Without a brewed python, brew install vim will use system's Python:

$ which python
==> /usr/bin/python
$ brew install vim # will link against system python
# system python has no `pip` shipped, so
$ sudo easy_install pip
$ pip install --user git+git://github.com/Lokaltog/powerline
@Lokaltog

This comment has been minimized.

Copy link
Member

commented Aug 14, 2013

Allright, I guess I've misunderstood some of the details surrounding this issue. Your instructions are clear so I think I'll update the OS X installation docs with those instead.

@samueljohn

This comment has been minimized.

Copy link

commented Aug 14, 2013

Cool, thanks!

@oryband

This comment has been minimized.

Copy link
Author

commented Aug 14, 2013

So, what is the proposed fix? For now I still have a ~/.pydistutils.cfg which causes brew doctor to complain, but allows me to install powerline properly.

@samueljohn

This comment has been minimized.

Copy link

commented Aug 14, 2013

@oryband if you have a brewed python you can just leave out the --user part and still don't need sudo.

@oryband

This comment has been minimized.

Copy link
Author

commented Aug 15, 2013

@samueljohn just updated brew, removed ~/.pydistutils.cfg and installed powerline without --user and I can confirm this works.

Thanks!

Please updated this in the docs and we can close this issue.

davidcelis added a commit to davidcelis/powerline that referenced this issue Aug 15, 2013

Update OSX installation instructions
If Python was installed via `brew` rather than `port`, the `--user` flag causes installation errors as described in powerline#631. Homebrew team member @samueljohn mentions this in powerline#552
@derekbrokeit

This comment has been minimized.

Copy link

commented Sep 10, 2013

Just to point out for those saying this is OSX specific, it is not. It appears to be something with homebrew's installation of python. I ran into this error for multiple python packages. I have python installed using linuxbrew (homebrew for linux). (Note: I have modified the python installation formula for linux by stripping out all OSX/Framework specific components).

@samueljohn is correct that installation works flawlessly without --user, but in my particular case I need all packages installed as --user because my system has multiple python instances of python2.7. I have both homebrew python and a proprietary software package that essentially is a custom python distribution (incomplete and thus not linkable). Therefore, I need both. I fixed it by simply relying on the homebrew version of pip and making ~/.local/lib/python2.7/site-packages a soft-link to the homebrew-python site-packages directory. Not elegant, but worked. It also saves me time in the end.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.