Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Six issue when installing package #3165

Closed
securitybites opened this issue Oct 6, 2015 · 93 comments
Closed

Six issue when installing package #3165

securitybites opened this issue Oct 6, 2015 · 93 comments

Comments

@securitybites
Copy link

@securitybites securitybites commented Oct 6, 2015

When trying to install awscli I get the following error.

admins-MacBook-Pro-5:~ user$ pip install awscli
Collecting awscli
  Using cached awscli-1.8.9-py2.py3-none-any.whl
Collecting botocore==1.2.7 (from awscli)
  Using cached botocore-1.2.7-py2.py3-none-any.whl
Collecting rsa<=3.3.0,>=3.1.2 (from awscli)
  Using cached rsa-3.2-py2.py3-none-any.whl
Collecting colorama<=0.3.3,>=0.2.5 (from awscli)
Collecting docutils>=0.10 (from awscli)
Requirement already satisfied (use --upgrade to upgrade): jmespath<1.0.0,>=0.7.1 in /Library/Python/2.7/site-packages (from botocore==1.2.7->awscli)
Collecting python-dateutil<3.0.0,>=2.1 (from botocore==1.2.7->awscli)
  Using cached python_dateutil-2.4.2-py2.py3-none-any.whl
Collecting pyasn1>=0.1.3 (from rsa<=3.3.0,>=3.1.2->awscli)
  Using cached pyasn1-0.1.9-py2.py3-none-any.whl
Collecting six>=1.5 (from python-dateutil<3.0.0,>=2.1->botocore==1.2.7->awscli)
  Using cached six-1.9.0-py2.py3-none-any.whl
Installing collected packages: six, python-dateutil, docutils, botocore, pyasn1, rsa, colorama, awscli
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/Library/Python/2.7/site-packages/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/var/folders/6t/h404bjcd5tb_4q86tpv_251rv_0h0j/T/pip-sYsqDS-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

Other user have mentioned the issue and point to pip:

aws/aws-cli#1522

https://bitbucket.org/gutworth/six/issues/134/uninstall-error-operation-not-permitted

@dstufft
Copy link
Member

@dstufft dstufft commented Oct 6, 2015

You're on El-Capitan right?

@Moeriki
Copy link

@Moeriki Moeriki commented Oct 6, 2015

I have this issue when installing AWS-SDK, which depends on six.

On El-Capitan indeed.

@dstufft
Copy link
Member

@dstufft dstufft commented Oct 6, 2015

This is because OS X El Capitan ships with six 1.4.1 installed already and when it attempts to uninstall it (because awscli depends on botocore, botocore depends on python-dateutil, and python-dateutil depends on six >= 1.5) it doesn't have permission to do so because System Integrity Protection doesn't allow even root to modify those directories.

Ideally, pip should just skip uninstalling those items since they aren't installed to site-packages they are installed to a special Apple directory. However, even if pip skips uninstalling those items and installs six into site-packages we'll hit another bug where Apple puts their pre-installed stuff earlier in the sys.path than site-packages. I've talked to Apple about this and I'm not sure if they're going to do anything about it or not.

@arubdesu
Copy link

@arubdesu arubdesu commented Oct 8, 2015

This affects many other packages which rely on Six, pardon my obliviousness, but is there any way to tell pip to either 1. install an updated version to /Library's site-packages and be pointed to prefer that one (instead of saying 'requirement satisfied! and exiting) or 2. ignore the version found in /System on 10.11+?

@dstufft
Copy link
Member

@dstufft dstufft commented Oct 8, 2015

pip install --ignore-installed six

@arubdesu
Copy link

@arubdesu arubdesu commented Oct 8, 2015

$ sudo -H pip install --ignore-installed six
Collecting six
  Using cached six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
Successfully installed six-1.4.1

Sure, but pip won't then prefer that upgraded version of Six when performing sudo installs:

sudo -H pip install mock
Requirement already satisfied (use --upgrade to upgrade): mock in /Library/Python/2.7/site-packages
Requirement already satisfied (use --upgrade to upgrade): funcsigs in /Library/Python/2.7/site-packages (from mock)
Requirement already satisfied (use --upgrade to upgrade): pbr>=0.11 in /Library/Python/2.7/site-packages (from mock)
Collecting six>=1.7 (from mock)
  Downloading six-1.10.0-py2.py3-none-any.whl
Installing collected packages: six
  Found existing installation: six 1.4.1
    DEPRECATION: Uninstalling a distutils installed project (six) has been deprecated and will be removed in a future version. This is due to the fact that uninstalling a distutils project will only partially uninstall the project.
    Uninstalling six-1.4.1:
Exception:
Traceback (most recent call last):
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/basecommand.py", line 211, in main
    status = self.run(options, args)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/commands/install.py", line 311, in run
    root=options.root_path,
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_set.py", line 640, in install
    requirement.uninstall(auto_confirm=True)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_install.py", line 716, in uninstall
    paths_to_remove.remove(auto_confirm)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/req/req_uninstall.py", line 125, in remove
    renames(path, new_path)
  File "/Library/Python/2.7/site-packages/pip-7.1.2-py2.7.egg/pip/utils/__init__.py", line 315, in renames
    shutil.move(old, new)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 302, in move
    copy2(src, real_dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 131, in copy2
    copystat(src, dst)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 103, in copystat
    os.chflags(dst, st.st_flags)
OSError: [Errno 1] Operation not permitted: '/tmp/pip-sy_6uC-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'
@arubdesu
Copy link

@arubdesu arubdesu commented Oct 8, 2015

...
Sorry about that, for some reason it's working after... running it a second time? Needed to --ignore-installed mock as well.

@arubdesu
Copy link

@arubdesu arubdesu commented Oct 8, 2015

Ah, so this is less a pip issue to address (although it seems like --ignore-installed is going to get very popular) than the user changing their path appropriately. (You'll just get all the support requests. Thanks, Apple!)
As per http://peak.telecommunity.com/DevCenter/setuptools#development-mode you can set a .pth file system-wide to make sure that directory is consulted first. I'll compose a radar to Apple so they can remove their heads from their butts about this.

@arubdesu
Copy link

@arubdesu arubdesu commented Oct 9, 2015

...Shouldn't there be some other, more permanent fix? easy_install has a .pth file, so there is some precedent here. I've documented what I think is the best near-term solution here: http://apple.stackexchange.com/a/210021/216

@reganjohnson
Copy link

@reganjohnson reganjohnson commented Oct 9, 2015

THANKS! The following allowed it to install correctly.

pip install awsebcli --upgrade --ignore-installed six

Running on OSX El Capitan

$ eb --version
EB CLI 3.5.4 (Python 2.7.1)

@dstufft
Copy link
Member

@dstufft dstufft commented Oct 9, 2015

@arubdesu I'm hoping the more permanent fix will be to have Apple just fix this, which I am talking to them about.

@propupul
Copy link

@propupul propupul commented Oct 12, 2015

Hello all!
I came across the same problem(thank goodness for this post!) while trying to install virtualenvwrapper. As all of you, and lastly @reganjohnson suggested --upgrade --ignore-installed six will work.
I did
sudo pip install virtualenvwrapper --upgrade --ignore-installed six

Running El Capitan as well

P.S: Please forgive me if I'm not posting this correctly I'm a complete noob to github :(

@nalindak
Copy link

@nalindak nalindak commented Oct 14, 2015

sudo pip install virtualenvwrapper --upgrade --ignore-installed six

Did work for me! thanks guys

@mattdeboard
Copy link

@mattdeboard mattdeboard commented Oct 23, 2015

You can disable System Integrity Protection completely via running csrutil disable in recovery mode. Do at your own risk etc. etc.

@arubdesu
Copy link

@arubdesu arubdesu commented Oct 23, 2015

Anyone can respond to this issue however they want, (playing chicken with Apple over this is like waiting for an updated version of rsync, in my opinion...) but if you're coming to this thread now, just know that you can work around this issue by:

  1. installing with pip to your user folder *
  2. installing with pip and --ignoring-installed *
  3. using a virtualenv
  4. installing your own python, e.g. with homebrew
    And after all of those options? Sure, you can TEMPORARILY disable SIP during the time you pip upgrade something in the System folder, but then you're also messing with the version Apple expects to be present. You can even "at your own risk etc. etc." disable, reboot, then enable --without fs to permanently stop just the filesystem protections. But I don't think you should get used to employing SIP-related fixes for python package installs, if anything the SIP-disabling options will go away, not get more lenient, so dealing with this new situation sooner rather than later seems wise.
    * also then possibly requiring you to employ a .pth file to ensure python picks up on your desired lookup order, as previously mentioned about sys.path
@mattdeboard
Copy link

@mattdeboard mattdeboard commented Oct 23, 2015

Just a suggestion friend, no need to get alarmed.

@ftiff
Copy link

@ftiff ftiff commented Oct 23, 2015

I agree with @arubdesu
Messing with system has never been a good idea.
Just use any of the other possibilities. And it's easier.

@arubdesu
Copy link

@arubdesu arubdesu commented Oct 23, 2015

And I agree with @dstufft that Apple should hard-code any of their python package dependencies to the System folder and allow the default path to check /Library before /System. (Not that I'm putting words in his mouth, just saying that's what I filed a radar for. http://www.openradar.me/radar?id=6192110889861120)

@monte-hayward
Copy link

@monte-hayward monte-hayward commented Oct 27, 2015

both of these result in six version 1.41:
pip install --ignore-installed six
pip install awsebcli --upgrade --ignore-installed six
El Capitan must go down with the ship...

@dstufft
Copy link
Member

@dstufft dstufft commented Oct 28, 2015

@monte-hayward Try pip install --user awsebcli (might need a --upgrade flag too).

@ftiff
Copy link

@ftiff ftiff commented Oct 29, 2015

Hi folks, FYI:
Engineering has determined that your bug report (23234649) is a duplicate of another issue (20471644) and will be closed.

If you fill a bug report please mention Radar # 20471644

@DogDogGoose
Copy link

@DogDogGoose DogDogGoose commented Oct 30, 2015

I found the easiest solution was to use homebrew to install a local version of python (thus pip). This avoids the complexities of El Capitan's protected install of python and instead installs python into /usr/local/bin. The various solutions for the "--ignore-installed six" did not work on my system.

Just do:

brew install python

The pip install command should work fine after that.

More information is available here:
http://apple.stackexchange.com/questions/209572/how-to-use-pip-after-the-el-capitan-max-os-x-upgrade

@owendall
Copy link

@owendall owendall commented Nov 12, 2015

Thanks much for this:
$ sudo pip install awscli --upgrade --ignore-installed six

@othyn
Copy link

@othyn othyn commented Nov 15, 2017

Yep, still an issue in High Sierra, 10.13.1.

Solved with: sudo pip install awscli --upgrade --ignore-installed six

@ileend
Copy link

@ileend ileend commented Dec 24, 2017

to install pytest on mac with el capitan, this worked:

sudo pip install pytest --upgrade --ignore-installed six

"Installing collected packages: py, attrs, pluggy, funcsigs, six, setuptools, pytest
Running setup.py install for pluggy ... done
Successfully installed attrs-17.3.0 funcsigs-1.0.2 pluggy-0.6.0 py-1.5.2 pytest-3.3.1 setuptools-38.2.4 six-1.11.0"
then to verify installation:

pytest --version

"This is pytest version 3.3.1, imported from /Library/Python/2.7/site-packages/pytest.pyc"

@ShatishPandey
Copy link

@ShatishPandey ShatishPandey commented Jan 22, 2018

Fixed installing IPython on MacOS High Sierra:

For me, using 'pip3' instead of 'pip' worked. Probable reason was pip was trying to deal with the default python installation provided by Mac. However I have installed Python3 using brew. So the following worked for me:

$ sudo pip3 install ipython[All]

@Ivoz
Copy link
Member

@Ivoz Ivoz commented Jan 22, 2018

@ShatishPandey to use pip with a specific python, you can always incant:

$ {mypython} -m pip (install/list/etc...)

Where {mypython} is python3 or /usr/bin/local/python2.5 or whatever else.

@davidwjx
Copy link

@davidwjx davidwjx commented Jan 25, 2018

@dstufft Thaaanks.

pip install --user jupyter

works fine in High Sierra, 10.13.3.

@codedogfish
Copy link

@codedogfish codedogfish commented Mar 28, 2018

Just do not use the system python, let pyenv save your world.

kislyuk added a commit to HumanCellAtlas/dcp-preview that referenced this issue Apr 5, 2018
kislyuk added a commit to HumanCellAtlas/dcp-preview that referenced this issue Apr 5, 2018
@rupav
Copy link

@rupav rupav commented May 15, 2018

@codedogfish can you explain, I am facing similar issue on travis build, but activating virtualenv not doing anything different on the build. Also its not possible to use --ignore-installed command for all packages. I was getting similar error for other package too, so I tihnk --ignore-installed is needed too.
Similar issue raised in:
https://stackoverflow.com/questions/49932759/pip-10-and-apt-how-to-avoid-cannot-uninstall-x-errors-for-distutils-packages

@cod3fran
Copy link

@cod3fran cod3fran commented Dec 14, 2018

pip install --ignore-installed six

thanks !!

@arjangch
Copy link

@arjangch arjangch commented May 19, 2019

I have the same problem in Mac Mojave right now.
pip install --ignore-installed six didn't help either

@lock lock bot added the S: auto-locked label Jun 18, 2019
@lock lock bot locked as resolved and limited conversation to collaborators Jun 18, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.