Mac install failure (Python errors) #1196

Open
mzsanford opened this Issue Oct 29, 2015 · 15 comments

Projects

None yet

7 participants

@mzsanford

I have a Mac running OS X 10.10.5 with Homebrew installed already. I attempted install via git clone and the ./letsencrypt-auto with the following results:

$ ./letsencrypt-auto
Bootstrapping dependencies for Mac OS X...
WARNING: Mac support is very experimental at present...
Warning: augeas-1.4.0 already installed
Warning: dialog-1.2-20150528 already installed
virtualenv Not Installed
Installing with pip
Requirement already satisfied (use --upgrade to upgrade): virtualenv in /Users/matt/Library/Python/2.7/lib/python/site-packages
Cleaning up...
Creating virtual environment...
./letsencrypt-auto: line 69: virtualenv: command not found

This was on master at the time, SHA 8e7c8ad. The bootstrap/mac.sh script uses hash virtualenv to test for the existence of the virtualenv package, which returns:

$ hash virtualenv
-bash: hash: virtualenv: not found

But then attempts an install via pip that returns:

$ pip install virtualenv
Requirement already satisfied (use --upgrade to upgrade): virtualenv in /Users/matt/Library/Python/2.7/lib/python/site-packages
Cleaning up...

I upgraded virtualenv as the command suggested (sudo pip install --upgrade virtualenv) and then started getting:

$ ./letsencrypt-auto
Bootstrapping dependencies for Mac OS X...
WARNING: Mac support is very experimental at present...
Warning: augeas-1.4.0 already installed
Warning: dialog-1.2-20150528 already installed
Creating virtual environment...
Traceback (most recent call last):
  File "/bin/virtualenv", line 7, in <module>
    from virtualenv import main
ImportError: No module named virtualenv

I assume this is somehow related to Python path issues (System vs Homebrew installations). As a non-Python developer attempt to use this tool for my non-Python project I'm not sure how to proceed. I also figured this were one of the kinks that would need to be ironed out before general release, hence filing the issue.

I'll move on to running it one Docker, but on the Mac that means via boot2docker and virtualization so that's hardly a wide audience solution.

@pde
Member
pde commented Oct 30, 2015

Looks like #963 didn't pan out perfectly. @douglasjose do you think you can help with this?

@pde
Member
pde commented Oct 30, 2015

Oh, actually, I have one idea: is there by any chance a python-virtualenv package that you can install?

@mzsanford

Homebrew has a pyenv-virtualenv package, but installing that (brew install pyenv-virtualenv) that does not change the problem. The /Users/matt/Library/… in the output makes me very suspicious because pip is installed via Homebrew in /usr/local. That is what makes me thing it's some path issue with my multiple Pythons:

$ which -a python
/usr/local/bin/python
/usr/local/bin/python
/usr/bin/python
@douglasjose
Contributor

@pde , yes, I'll have a look.

@pde pde added this to the 1.0 for launch milestone Oct 30, 2015
@kuba
Contributor
kuba commented Oct 31, 2015
@douglasjose
Contributor

@mzsanford , I believe I managed to replicate your issue locally. It seems that a previous virtualenv installation is preventing pip install virtualenv from installing successfully.

If I undo the local changes made by bootstrap/mac.sh (with pip uninstall virtualenv ; brew uninstall python) and manually install virtualenv using easy_install

easy_install --user virtualenv

it installs virtualenv dependencies to the Library directory in my home dir:

Installed /Users/douglas/Library/Python/2.7/lib/python/site-packages/virtualenv-13.1.2-py2.7.egg
Processing dependencies for virtualenv
Finished processing dependencies for virtualenv

But no executable seems to be available in the path

$ which virtualenv
virtualenv not found

I was able to work it around by having pip uninstalling the virtualenv installation managed by easy_install and installing it again.

$ pip uninstall virtualenv
Uninstalling virtualenv-13.1.2:
/Users/douglas/Library/Python/2.7/lib/python/site-packages/virtualenv-13.1.2-py2.7.egg
Proceed (y/n)? y
Successfully uninstalled virtualenv-13.1.2

After that you should be able to run bootstrap/mac.sh and the virtualenv command should be available in your path.

Would you be able to please test this workaround?

@pde , @kuba

With regards to the completeness of #963 , provided my theory about the root cause of the issue is correct, I'm not sure if there is much we can do, as a previous Python installation is interfering with the dependencies. I don't think it would be safe to code a script that may be interfering with an existing setup that may be used for something else.

What I could do is to add a last check for the presence of virtualenv in the path and display a message warning that the pip install virtualenv didn't finish as we expected, and they should manually check any previous virtualenv installations.

What do you think?

@BKreisel
Contributor
BKreisel commented Nov 4, 2015

Another option might be adding the default OS X site-packages directory to the path or a pip --user directory. That would only work until the reboot and feels a bit hacky.

We could also put a check in that looks for the default installation that asks the user to either permanently add to path or replace it with a pip version or something. Neither of these sound like very clean solutions though.

@pde
Member
pde commented Nov 4, 2015

Could letsencrypt-auto on Mac add these directories (OS X site packages and/or ~/Library) to PATH / PYTHONPATH if virtualenv appears to be installed in one of them?

@mzsanford

I attempted pip uninstall but it said it was not installed:

$ pip uninstall virtualenv
Cannot uninstall requirement virtualenv, not installed
Storing debug log for failure in /Users/matt/.pip/pip.log

I was able to run the easy_install like so:

$ easy_install --user virtualenv
Searching for virtualenv
Reading https://pypi.python.org/simple/virtualenv/
Best match: virtualenv 13.1.2
Downloading https://pypi.python.org/packages/source/v/virtualenv/virtualenv-13.1.2.tar.gz#md5=b989598f068d64b32dead530eb25589a
Processing virtualenv-13.1.2.tar.gz
Writing /var/folders/bm/d4g3ncms3f91f1y90djvthbw0000gn/T/easy_install-zLCEPD/virtualenv-13.1.2/setup.cfg
Running virtualenv-13.1.2/setup.py -q bdist_egg --dist-dir /var/folders/bm/d4g3ncms3f91f1y90djvthbw0000gn/T/easy_install-zLCEPD/virtualenv-13.1.2/egg-dist-tmp-Lu7Rk3
warning: no previously-included files matching '*' found under directory 'docs/_templates'
warning: no previously-included files matching '*' found under directory 'docs/_build'
Adding virtualenv 13.1.2 to easy-install.pth file
Installing virtualenv script to /Users/matt/Library/Python/2.7/bin
Installing virtualenv-2.7 script to /Users/matt/Library/Python/2.7/bin

Installed /Users/matt/Library/Python/2.7/lib/python/site-packages/virtualenv-13.1.2-py2.7.egg
Processing dependencies for virtualenv
Finished processing dependencies for virtualenv

and the bootstrap/mac.sh script now succeeds but letsencrypt-auto does not:

$ ./letsencrypt-auto
-n Updating letsencrypt and virtual environment dependencies...
./letsencrypt-auto: line 85: /Users/matt/.local/share/letsencrypt/bin/pip: No such file or directory

Perhaps this is just more PATH madness:

$ which -a virtualenv
/bin/virtualenv

but even adding /Users/matt/Library/Python/2.7/bin to the front of my PATH does not help. I uninstalled python via Homebrew but no change. Still the error about /Users/matt/.local/share/letsencrypt/bin/pip. The error has changed but I still don't feel closer to success. Any other thoughts?

@pde
Member
pde commented Nov 4, 2015

@mzsanford it sounds like your situation is slightly different than the (similar) reproduction that @douglasjose created. We don't know if your copy of /bin/virtualenv came from easyinstall or from some previous install by other means.

I'm going to try a simple test version of my suggestion above; will send you instructions for running it in a second.

@pde
Member
pde commented Nov 4, 2015

OK here's an attempt to make letsencrypt-auto a bit smarter about finding virutalenv.

@mzsanford , can you try this: (first cd to wherever your git checkout of letsencrypt is)

rm -rf /Users/matt/.local/share/letsencrypt
git remote update
git checkout mac-install # should create a new branch to track origin/mac-install
./letsencrypt-auto --verbose
@pde
Member
pde commented Nov 4, 2015

We may also need to add things to PYTHONPATH, so only 50/50 that the above attempt will work.

@pde pde added a commit that referenced this issue Nov 4, 2015
@pde pde Require --debug for letsencrypt-auto on platforms thare aren't regula…
…rly tested

Mitigates: #1270 #1196
d2f1ae9
@pde
Member
pde commented Nov 6, 2015

Since we now require Mac users to run with --debug, and disclaim that Mac support is experimental, I think we can kick this out of the Public Beta milestone.

@pde pde modified the milestone: Nice for 1.0, 1.0 for launch Nov 6, 2015
@egillespie

With OS X 10.11.1 (El Capitan), Homebrew already installed, and no existing virtualenv installed, I was able to get letsencrypt-auto --debug to run successfully by running the following two commands first:

$ sudo chown -R $(whoami):admin /usr/local
$ sudo chown -R $(whoami):admin /Library/Python/2.7/site-packages/

Since upgrading to El Capitan, I find myself having to re-run sudo chown -R $(whoami):admin /usr/local after each OS update if I want to install anything with Homebrew. I'm not sure if this is the best solution, but it has worked well for me so far when I get errors such as:

Error: The `brew link` step did not complete successfully
The formula built, but is not symlinked into /usr/local
Could not symlink bin/augparse
/usr/local/bin is not writable.

I applied the same tactic to /Library/Python/2.7/site-packages when I later got the error:

IOError: [Errno 13] Permission denied: '/Library/Python/2.7/site-packages/virtualenv.py'
@vvakame
vvakame commented Apr 29, 2016

I got an error above.

$ ./letsencrypt-auto --debug -v                                                                                                                 1 ↵
Bootstrapping dependencies via Mac OS X...
Using Homebrew to install dependencies...
Warning: augeas-1.4.0 already installed
Warning: dialog-1.3-20160209 already installed
Checking for new version...
Creating virtual environment...
Traceback (most recent call last):
  File "/usr/local/bin/virtualenv", line 7, in <module>
    from virtualenv import main
ImportError: No module named 'virtualenv'

I exec above command.

$ pip3 install virtualenv

I can see Let's encrypt wizard.

FYI, my environment.

$ git rev-parse HEAD
eb56c166d76c15f5b886f391e49532478ce63d40
$ brew list
ant         eigen           ilmbase         libxml2         pcre            tree
augeas          ffmpeg          isl         maven           peco            wget
autoconf        gcc         jpeg            mpfr            pkg-config      x264
brew-cask       gdbm            jq          nodebrew        python3         xvid
cmake           git         lame            numpy           rbenv           xz
curl            gmp         libmpc          oniguruma       readline        zsh
dialog          gnu-sed         libpng          opencv3         ruby-build      zsh-completions
docker          go          libtiff         openexr         sqlite
docker-machine      hub         libvo-aacenc        openssl         the_silver_searcher
$  pip3 list
fluent-logger (0.4.0)
msgpack-python (0.4.7)
nose (1.3.7)
numpy (1.11.0)
oauthlib (1.0.3)
Pillow (3.1.1)
pip (8.1.1)
requests (2.9.1)
requests-oauthlib (0.6.1)
setuptools (20.2.2)
slackweb (1.0.5)
tornado (4.3)
u-msgpack-python (2.1)
urllib3 (1.14)
virtualenv (15.0.1)
wheel (0.26.0)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment