virtualenv fails to create virtual environment from Homebrew-installed Python 3.4.3 #779

Closed
aarzee opened this Issue Aug 1, 2015 · 11 comments

Projects

None yet

6 participants

@aarzee
aarzee commented Aug 1, 2015

When I try to create a virtual environment with Python 3.4.3, virtualenv errors out:

Running virtualenv with interpreter /usr/local/bin/python3.4
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/virtualenv.py", line 23, in <module>
    import subprocess
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/subprocess.py", line 405, in <module>
    import _posixsubprocess
ImportError: dlopen(/usr/local/lib/python2.7/site-packages/_posixsubprocess.so, 2): Symbol not found: _PyString_AsString
  Referenced from: /usr/local/lib/python2.7/site-packages/_posixsubprocess.so
  Expected in: flat namespace
 in /usr/local/lib/python2.7/site-packages/_posixsubprocess.so

I've found another reference to the exact same error on StackOverflow, with the only answer suggesting to add the --no-site-packages argument. Looking through virtualenv -h, I see that not only is it deprecated, it doesn't change the setting, as its functionality is currently the default. Predictably, this does not help.

@tdsmith
tdsmith commented Aug 1, 2015

What's your PYTHONPATH look like? It looks like the problem is that your python 3 is loading modules built for python 2.7.

@ionelmc
ionelmc commented Aug 1, 2015

Seems similar to #705

@aarzee
aarzee commented Aug 1, 2015

Doing echo $PYTHONPATH resulted in a blank line.

@Ivoz Ivoz added a commit to Ivoz/virtualenv that referenced this issue Oct 2, 2015
@Ivoz Ivoz Remove file's directory from sys.path ASAP
This is needed to particularly when a new interpreter is used,
via -p/--python. Re-execing the same `virtualenv.py` will generally lead
to its path being added to the start of sys.path (as usual). And usually
its path will be the site-packages of the previous interpreter. This
will lead to issues if older backported packages are present in the
old environment (which will then get preference being imported).

Should fix #779, #774, #763
8925870
@Ivoz
Member
Ivoz commented Oct 2, 2015

@aarzee would appreciate if you could check if above commit solves the issue

https://github.com/Ivoz/virtualenv/archive/early-syspath-removal.zip

@Ivoz Ivoz added a commit to Ivoz/virtualenv that referenced this issue Oct 2, 2015
@Ivoz Ivoz Remove file's directory from sys.path ASAP
This is needed to particularly when a new interpreter is used,
via -p/--python. Re-execing the same `virtualenv.py` will generally lead
to its path being added to the start of sys.path (as usual). And usually
its path will be the site-packages of the previous interpreter. This
will lead to issues if older backported packages are present in the
old environment (which will then get preference being imported).

Should fix #779, #774, #763
832c515
@aarzee
aarzee commented Oct 2, 2015

Unfortunately, yesterday I upgraded to El Capitan, and pyenv is failing to install 3.4.3 (which is now unavailable via Homebrew). I can't reproduce the bug with current PyPI virtualenv when making a Python 3.5 environment.

@Ivoz Ivoz added a commit to Ivoz/virtualenv that referenced this issue Oct 17, 2015
@Ivoz @Ivoz Ivoz + Ivoz Remove file's directory from sys.path ASAP
This is needed particularly when a new interpreter is used,
via -p/--python. Re-execing the same `virtualenv.py` will generally lead
to its path being added to the start of sys.path (as usual). And usually
its path will be the site-packages of the previous interpreter. This
will lead to issues if older backported packages are present in the
old environment (which will then get preference being imported).

Should fix #779, #774, #763
348f80c
@Ivoz Ivoz closed this in #805 Oct 17, 2015
@vmarquet
vmarquet commented Sep 26, 2016 edited

I'm having the exact same issue, can someone please reopen?

$ virtualenv -p /usr/local/bin/python3 test
Running virtualenv with interpreter /usr/local/bin/python3
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/virtualenv.py", line 27, in <module>
    import subprocess
  File "/usr/local/Cellar/python3/3.4.3_2/Frameworks/Python.framework/Versions/3.4/lib/python3.4/subprocess.py", line 405, in <module>
    import _posixsubprocess
ImportError: dlopen(/Library/Python/2.7/site-packages/_posixsubprocess.so, 2): Symbol not found: _PyString_AsString
  Referenced from: /Library/Python/2.7/site-packages/_posixsubprocess.so
  Expected in: flat namespace
 in /Library/Python/2.7/site-packages/_posixsubprocess.so

Other details:

  • Mac OS X, 10.11.6
  • $ echo $PYTHONPATH => /Library/Python/2.7/site-packages:
  • Python3 installed with brew:
$ brew info python3
python3: stable 3.5.2 (bottled), devel 3.6.0a3, HEAD
Interpreted, interactive, object-oriented programming language
https://www.python.org/
/usr/local/Cellar/python3/3.4.3_2 (4,773 files, 69.2M)
  Poured from bottle on 2015-08-11 at 17:58:19
From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/python3.rb
==> Dependencies
Build: xz ✔, pkg-config ✔, sphinx-doc ✘
Required: openssl ✔
Recommended: readline ✔, sqlite ✔, gdbm ✔, xz ✔
Optional: homebrew/dupes/tcl-tk ✘, sphinx-doc ✘
==> Options
--universal
    Build a universal binary
--with-quicktest
    Run `make quicktest` after the build
--with-sphinx-doc
    Build HTML documentation
--with-tcl-tk
    Use Homebrew's Tk instead of OS X Tk (has optional Cocoa and threads support)
--without-gdbm
    Build without gdbm support
--without-readline
    Build without readline support
--without-sqlite
    Build without sqlite support
--without-xz
    Build without xz support
--devel
    Install development version 3.6.0a3
--HEAD
    Install HEAD version
==> Caveats
Pip, setuptools, and wheel have been installed. To update them
  pip3 install --upgrade pip setuptools wheel

You can install Python packages with
  pip3 install <package>

They will install into the site-package directory
  /usr/local/lib/python3.5/site-packages

See: https://github.com/Homebrew/brew/blob/master/share/doc/homebrew/Homebrew-and-Python.md
@vmarquet
vmarquet commented Sep 26, 2016 edited

Problem solved, it was due to Homebrew. It was solved with the following commands:

$ brew unlink python3
$ brew link --overwrite python3
$ unset PYTHONPATH

After that, virtualenv worked as expected.

@tdsmith
tdsmith commented Sep 26, 2016

unset PYTHONPATH should have been sufficient; Homebrew didn't contribute
to the problem.

On Mon, Sep 26, 2016, 09:22 Vincent Marquet notifications@github.com
wrote:

Problem solved, it was due to Homebrew. It was solved with the following
commands:

$ brew unlink python3
$ brew link --overwrite python3
$ unset PYTHONPATH

After that, virtualenv worked as expected.


You are receiving this because you commented.
Reply to this email directly, view it on GitHub
#779 (comment),
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAKnQcXE4gPz_iaEFNHC14YX9thiZwosks5qt_FTgaJpZM4Fj2fP
.

@vmarquet

No, I tried unset PYTHONPATH before and it didn't solved the problem. After unset PYTHONPATH, the error message was different, and related to homebrew problems.

@BenjaminHCCarr

Sadly I am still getting this error, command I am firing off:
benc$ virtualenv --no-site-packages --distribute --relocatable -p /usr/local/bin/python3 --always-copy .env

Error:

benc$ virtualenv --no-site-packages --distribute --relocatable -p /usr/local/bin/python3 --always-copy .env
Running virtualenv with interpreter /usr/local/bin/python3
The environment doesn't have a file /Users/benc/Code/git/Mine/oh-data-google-fit/.env/bin/activate_this.py -- please re-run virtualenv on this environment to update it
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/virtualenv.py", line 2328, in <module>
    main()
  File "/usr/local/lib/python2.7/site-packages/virtualenv.py", line 700, in main
    make_environment_relocatable(home_dir)
  File "/usr/local/lib/python2.7/site-packages/virtualenv.py", line 1588, in make_environment_relocatable
    fixup_scripts(home_dir, bin_dir)
  File "/usr/local/lib/python2.7/site-packages/virtualenv.py", line 1610, in fixup_scripts
    for filename in os.listdir(bin_dir):
FileNotFoundError: [Errno 2] No such file or directory: '/Users/benc/Code/git/Mine/oh-data-google-fit/.env/bin'

I tried unset PYTHONPATH; no luck, then tried:

benc$ brew unlink python3
Unlinking /usr/local/Cellar/python3/3.6.0... 23 symlinks removed
tyr:~/Code/git/Mine/oh-data-google-fit benc$ brew link --overwrite python3
Linking /usr/local/Cellar/python3/3.6.0... 23 symlinks created
tyr:~/Code/git/Mine/oh-data-google-fit benc$ unset PYTHONPATH

Same error.

tyr:~/Code/git/Mine/oh-data-google-fit benc$ which python3
/usr/local/bin/python3
tyr:~/Code/git/Mine/oh-data-google-fit benc$ ls -l `!!`
ls -l `which python3`
lrwxr-xr-x  1 benc  admin  35 Jan 23 14:16 /usr/local/bin/python3 -> ../Cellar/python3/3.6.0/bin/python3
tyr:~/Code/git/Mine/oh-data-google-fit benc$ echo $PYTHONPATH

tyr:~/Code/git/Mine/oh-data-google-fit benc$

Thoughts?

@BenjaminHCCarr

Okay, problem found, it was the --relocatable with Python3, maybe it was deprecated.

benc$ virtualenv --no-site-packages --distribute -p /usr/local/bin/python3 --always-copy .env_OH_GFit
Running virtualenv with interpreter /usr/local/bin/python3
Using base prefix '/usr/local/Cellar/python3/3.6.0/Frameworks/Python.framework/Versions/3.6'
New python executable in /Users/benc/Code/git/Mine/oh-data-google-fit/.env_OH_GFit/bin/python3.6
Also creating executable in /Users/benc/Code/git/Mine/oh-data-google-fit/.env_OH_GFit/bin/python
Installing setuptools, pip, wheel...done.

worked fine.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment