Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tox can't find python3.x executable in virtualenv #202

Closed
gangefors opened this issue Mar 3, 2017 · 16 comments
Closed

Tox can't find python3.x executable in virtualenv #202

gangefors opened this issue Mar 3, 2017 · 16 comments

Comments

@gangefors
Copy link

@gangefors gangefors commented Mar 3, 2017

After creating a virtualenv with:

pyenv virtualenv 3.5.3 myenv

I'm not able to run tox with this ini file:

[tox]
envlist = py35

[testenv]
commands =
    pytest {posargs: --pep8 --pylint}

deps = .[test]

The following error appears:

ERROR: InvocationError: Failed to get version_info for python3.5: b"pyenv: python3.5: command not found\n\nThe `python3.5' command exists in these Python versions:\n  3.5.3\n\n"

After checking my virtualenv it's clear that it doesn't contain a python3.5 symlink.

(myenv) ➜  ~ ll /home/stefanga/.pyenv/versions/myenv/bin/      
total 32K
-rw-r--r-- 1 stefanga axusers 2.2K Mar  3 13:06 activate
-rw-r--r-- 1 stefanga axusers 1.3K Mar  3 13:06 activate.csh
-rw-r--r-- 1 stefanga axusers 2.4K Mar  3 13:06 activate.fish
-rwxr-xr-x 1 stefanga axusers  276 Mar  3 13:06 easy_install
-rwxr-xr-x 1 stefanga axusers  276 Mar  3 13:06 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip3
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip3.5
lrwxrwxrwx 1 stefanga axusers   47 Mar  3 13:06 python -> /home/stefanga/.pyenv/versions/3.5.3/bin/python
lrwxrwxrwx 1 stefanga axusers    6 Mar  3 13:06 python3 -> python

After creating the python3.5 -> python symlink manually tox executed without any errors.

Shouldn't all executables (python, pythonx and python x.y) be available in the virtualenv?


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@blueyed
Copy link
Collaborator

@blueyed blueyed commented Mar 3, 2017

@gangefors
Copy link
Author

@gangefors gangefors commented Mar 6, 2017

Thanks! But should I really have to use another plugin to get the expected behaviour?

To me it just feels like there should be a pythonX.Y symlink in the virtualenv's bin folder after creating a new virtualenv. Since the executables are present in the system's python and also pyenv installed python versions I'd expect to find similar symlinks in the virtualenvs as well.

But if you disagree, feel free to close this issue.

@eddie-dunn
Copy link

@eddie-dunn eddie-dunn commented Mar 7, 2017

I'm with @gangefors on this. Is there a reason that said symlinks can't be created automatically by pyenv?

@yyuu
Copy link
Collaborator

@yyuu yyuu commented Mar 7, 2017

If you think so, just write something by yourself.

@yyuu yyuu closed this Mar 7, 2017
@gangefors
Copy link
Author

@gangefors gangefors commented Mar 7, 2017

To anyone that finds this thread and experience the same issue and don't want to use yet another plugin to solve it.

I've found a workaround to this issue.

By specifying exactly which python binary you want to use you can get the expected behaviour.

pyenv virtualenv -p python3.5 3.5.3 myenv

will create a virtualenv with the following contents in ./bin

ll ~/.pyenv/versions/myenv/bin 
total 12M
-rw-r--r-- 1 stefanga axusers 2.1K Mar  7 14:51 activate
-rw-r--r-- 1 stefanga axusers 1.1K Mar  7 14:51 activate.csh
-rw-r--r-- 1 stefanga axusers 2.2K Mar  7 14:51 activate.fish
-rw-r--r-- 1 stefanga axusers 1.2K Mar  7 14:51 activate_this.py
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python -> python3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python3 -> python3.5
-rwxr-xr-x 1 stefanga axusers  12M Mar  7 14:51 python3.5
-rwxr-xr-x 1 stefanga axusers 2.4K Mar  7 14:51 python-config
-rwxr-xr-x 1 stefanga axusers  256 Mar  7 14:51 wheel

The plugin (probably the python venv module) now creates a virtualenv with python3.5 binary as the primary binary and creates symlinks to this. The result is that I've got my pythonX.Y binary in the virtualenv and tox is happy again.

@anentropic
Copy link

@anentropic anentropic commented Oct 26, 2017

I have same problem:

$ pyenv virtualenv 3.6.2 myenv
$ pyenv activate myenv
$ tox -e py36-dj111
py36-dj111 create: /App/.tox/py36-dj111
ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)
_______________________________________________________ summary _______________________________________________________
ERROR:   py36-dj111: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)

I could not get tox-pyenv plugin to solve this issue (it seems to be intended to solve a slightly different problem?)

However @gangefors suggestion above does work

If I delete my virtualenv and recreate it with pyenv virtualenv -p python3.6 myenv then tox will work

So it seems to be down to pyenv-virtualenv not creating the python3.6 symlink. I note that pip3.6 and easy_install-3.6 links are created.

@eddie-dunn
Copy link

@eddie-dunn eddie-dunn commented Oct 27, 2017

@yyuu I would like to solve this, preferrably in the same place as where the python and python3 symlinks are created. I couldn't find where this is, though.

Could you point me in the right direction?

@gangefors
Copy link
Author

@gangefors gangefors commented Oct 30, 2017

Possibly related to #206

@madhurabhogate
Copy link

@madhurabhogate madhurabhogate commented Nov 29, 2017

@gangefors : The solution you suggested (pyenv virtualenv -p python3.5 3.5.3 myenv) works.
But in my tox.ini I want to verify for multiple python versions say py35, py36. By creating the the virtualenv in above manner; tox runs successfully for py35 version but fails for py36.

ERROR: py36: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\n\nThe `python3.6' command exists in these Python versions:\n 3.6.0\n 3.6.1\n\n"',)

How do we overcome this issues ? Can you suggest please ?

@gangefors
Copy link
Author

@gangefors gangefors commented Nov 29, 2017

@madhurabhogate I would suggest that you create different venvs for different python versions. Then you can use them at the same time with pyenv <shell|local> <py35venv> <py36venv>. The python binaries from both venvs should be available in your path for tox to find.

# Do the following in your shell
$ pyenv virtualenv -p python3.5 3.5.4 py35
$ pyenv virtualenv -p python3.6 3.6.3 py36
$ pyenv shell py36 py35

# Then you should have two active environments you can use
$ pyenv versions
  system
  3.5.4
  3.5.4/envs/py35
  3.6.3
  3.6.3/envs/py36
* py35 (set by PYENV_VERSION environment variable)
* py36 (set by PYENV_VERSION environment variable)

# Now you should be able to run tox for both py35 and py36 envlists
@madhurabhogate
Copy link

@madhurabhogate madhurabhogate commented Nov 29, 2017

@gangefors : Thank you. An example would be really helpful :)

@hhstore
Copy link

@hhstore hhstore commented Apr 7, 2018

you can try:

pyenv shell 3.5.3

ref: pyenv/pyenv#856

@bsolomon1124
Copy link

@bsolomon1124 bsolomon1124 commented Oct 25, 2018

Note that there is a small typo in @gangefors 's example (at least for pyenv 1.2.7) with a flag:

$ brew install pyenv-virtualenv
$ pyenv virtualenv -p python3.5 3.5.6 py35
$ pyenv virtualenv -p python3.6 3.6.6 py36
$ pyenv virtualenv -p python3.7 3.7.0 py37
$ pyenv shell py35 py36 py37

If using with tox, order of cmds seems to be a tiny bit finnicky and very particular:

$ pyenv shell py35 py36 py37
$ python3.7 -m venv venv
$ source ./venv/bin/activate
$ tox
@GhostofGoes
Copy link

@GhostofGoes GhostofGoes commented Jan 26, 2019

The issue for me was simply a case of not having the Pythons on the pyenv path. Fixed by adding them to the path:

pyenv local 3.7.2 3.6.8 3.5.6 pypy3.5-6.0.0 pypy2.7-6.0.0
@mark0978
Copy link

@mark0978 mark0978 commented Feb 13, 2019

I did pyenv local 3.4.9 3.5.6 3.6.8 but that did NOT fix the shims. For that I had to use rm (to remove the old shim) and ln -s to create the new shim

@cicorias
Copy link

@cicorias cicorias commented Feb 23, 2019

i was able to fix this in a docker build here
https://github.com/Microsoft/agogosml/blob/7a41e4d136534c9b8caf93dc1a87cb52a62aa845/agogosml_cli/Dockerfile

The "gist" of it all is below:

RUN curl https://pyenv.run | bash
#see https://www.loganasherjones.com/2018/03/setting-up-your-python-environment/

RUN pyenv install 3.5.6
RUN pyenv install 3.6.8
RUN pyenv install 3.7.2
RUN pip install pytest tox
RUN pip install tox-globinterpreter

WORKDIR /usr/src/agogosml_cli

RUN echo "3.5.6" >> .python-version
RUN echo "3.6.8" >> .python-version
RUN echo "3.7.2" >> .python-version
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.