New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tox can't find python3.x executable in virtualenv #202

Closed
gangefors opened this Issue Mar 3, 2017 · 12 comments

Comments

Projects
None yet
7 participants
@gangefors

gangefors commented Mar 3, 2017

After creating a virtualenv with:

pyenv virtualenv 3.5.3 myenv

I'm not able to run tox with this ini file:

[tox]
envlist = py35

[testenv]
commands =
    pytest {posargs: --pep8 --pylint}

deps = .[test]

The following error appears:

ERROR: InvocationError: Failed to get version_info for python3.5: b"pyenv: python3.5: command not found\n\nThe `python3.5' command exists in these Python versions:\n  3.5.3\n\n"

After checking my virtualenv it's clear that it doesn't contain a python3.5 symlink.

(myenv) ➜  ~ ll /home/stefanga/.pyenv/versions/myenv/bin/      
total 32K
-rw-r--r-- 1 stefanga axusers 2.2K Mar  3 13:06 activate
-rw-r--r-- 1 stefanga axusers 1.3K Mar  3 13:06 activate.csh
-rw-r--r-- 1 stefanga axusers 2.4K Mar  3 13:06 activate.fish
-rwxr-xr-x 1 stefanga axusers  276 Mar  3 13:06 easy_install
-rwxr-xr-x 1 stefanga axusers  276 Mar  3 13:06 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip3
-rwxr-xr-x 1 stefanga axusers  248 Mar  3 13:06 pip3.5
lrwxrwxrwx 1 stefanga axusers   47 Mar  3 13:06 python -> /home/stefanga/.pyenv/versions/3.5.3/bin/python
lrwxrwxrwx 1 stefanga axusers    6 Mar  3 13:06 python3 -> python

After creating the python3.5 -> python symlink manually tox executed without any errors.

Shouldn't all executables (python, pythonx and python x.y) be available in the virtualenv?


Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.

@blueyed

This comment has been minimized.

Show comment
Hide comment
@blueyed
Collaborator

blueyed commented Mar 3, 2017

@gangefors

This comment has been minimized.

Show comment
Hide comment
@gangefors

gangefors Mar 6, 2017

Thanks! But should I really have to use another plugin to get the expected behaviour?

To me it just feels like there should be a pythonX.Y symlink in the virtualenv's bin folder after creating a new virtualenv. Since the executables are present in the system's python and also pyenv installed python versions I'd expect to find similar symlinks in the virtualenvs as well.

But if you disagree, feel free to close this issue.

gangefors commented Mar 6, 2017

Thanks! But should I really have to use another plugin to get the expected behaviour?

To me it just feels like there should be a pythonX.Y symlink in the virtualenv's bin folder after creating a new virtualenv. Since the executables are present in the system's python and also pyenv installed python versions I'd expect to find similar symlinks in the virtualenvs as well.

But if you disagree, feel free to close this issue.

@eddie-dunn

This comment has been minimized.

Show comment
Hide comment
@eddie-dunn

eddie-dunn Mar 7, 2017

I'm with @gangefors on this. Is there a reason that said symlinks can't be created automatically by pyenv?

eddie-dunn commented Mar 7, 2017

I'm with @gangefors on this. Is there a reason that said symlinks can't be created automatically by pyenv?

@yyuu

This comment has been minimized.

Show comment
Hide comment
@yyuu

yyuu Mar 7, 2017

Collaborator

If you think so, just write something by yourself.

Collaborator

yyuu commented Mar 7, 2017

If you think so, just write something by yourself.

@yyuu yyuu closed this Mar 7, 2017

@gangefors

This comment has been minimized.

Show comment
Hide comment
@gangefors

gangefors Mar 7, 2017

To anyone that finds this thread and experience the same issue and don't want to use yet another plugin to solve it.

I've found a workaround to this issue.

By specifying exactly which python binary you want to use you can get the expected behaviour.

pyenv virtualenv -p python3.5 3.5.3 myenv

will create a virtualenv with the following contents in ./bin

ll ~/.pyenv/versions/myenv/bin 
total 12M
-rw-r--r-- 1 stefanga axusers 2.1K Mar  7 14:51 activate
-rw-r--r-- 1 stefanga axusers 1.1K Mar  7 14:51 activate.csh
-rw-r--r-- 1 stefanga axusers 2.2K Mar  7 14:51 activate.fish
-rw-r--r-- 1 stefanga axusers 1.2K Mar  7 14:51 activate_this.py
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python -> python3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python3 -> python3.5
-rwxr-xr-x 1 stefanga axusers  12M Mar  7 14:51 python3.5
-rwxr-xr-x 1 stefanga axusers 2.4K Mar  7 14:51 python-config
-rwxr-xr-x 1 stefanga axusers  256 Mar  7 14:51 wheel

The plugin (probably the python venv module) now creates a virtualenv with python3.5 binary as the primary binary and creates symlinks to this. The result is that I've got my pythonX.Y binary in the virtualenv and tox is happy again.

gangefors commented Mar 7, 2017

To anyone that finds this thread and experience the same issue and don't want to use yet another plugin to solve it.

I've found a workaround to this issue.

By specifying exactly which python binary you want to use you can get the expected behaviour.

pyenv virtualenv -p python3.5 3.5.3 myenv

will create a virtualenv with the following contents in ./bin

ll ~/.pyenv/versions/myenv/bin 
total 12M
-rw-r--r-- 1 stefanga axusers 2.1K Mar  7 14:51 activate
-rw-r--r-- 1 stefanga axusers 1.1K Mar  7 14:51 activate.csh
-rw-r--r-- 1 stefanga axusers 2.2K Mar  7 14:51 activate.fish
-rw-r--r-- 1 stefanga axusers 1.2K Mar  7 14:51 activate_this.py
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install
-rwxr-xr-x 1 stefanga axusers  277 Mar  7 14:51 easy_install-3.5
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3
-rwxr-xr-x 1 stefanga axusers  249 Mar  7 14:51 pip3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python -> python3.5
lrwxrwxrwx 1 stefanga axusers    9 Mar  7 14:51 python3 -> python3.5
-rwxr-xr-x 1 stefanga axusers  12M Mar  7 14:51 python3.5
-rwxr-xr-x 1 stefanga axusers 2.4K Mar  7 14:51 python-config
-rwxr-xr-x 1 stefanga axusers  256 Mar  7 14:51 wheel

The plugin (probably the python venv module) now creates a virtualenv with python3.5 binary as the primary binary and creates symlinks to this. The result is that I've got my pythonX.Y binary in the virtualenv and tox is happy again.

@anentropic

This comment has been minimized.

Show comment
Hide comment
@anentropic

anentropic Oct 26, 2017

I have same problem:

$ pyenv virtualenv 3.6.2 myenv
$ pyenv activate myenv
$ tox -e py36-dj111
py36-dj111 create: /App/.tox/py36-dj111
ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)
_______________________________________________________ summary _______________________________________________________
ERROR:   py36-dj111: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)

I could not get tox-pyenv plugin to solve this issue (it seems to be intended to solve a slightly different problem?)

However @gangefors suggestion above does work

If I delete my virtualenv and recreate it with pyenv virtualenv -p python3.6 myenv then tox will work

So it seems to be down to pyenv-virtualenv not creating the python3.6 symlink. I note that pip3.6 and easy_install-3.6 links are created.

anentropic commented Oct 26, 2017

I have same problem:

$ pyenv virtualenv 3.6.2 myenv
$ pyenv activate myenv
$ tox -e py36-dj111
py36-dj111 create: /App/.tox/py36-dj111
ERROR: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)
_______________________________________________________ summary _______________________________________________________
ERROR:   py36-dj111: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\\n\\nThe `python3.6\' command exists in these Python versions:\\n  3.6.2\\n\\n"',)

I could not get tox-pyenv plugin to solve this issue (it seems to be intended to solve a slightly different problem?)

However @gangefors suggestion above does work

If I delete my virtualenv and recreate it with pyenv virtualenv -p python3.6 myenv then tox will work

So it seems to be down to pyenv-virtualenv not creating the python3.6 symlink. I note that pip3.6 and easy_install-3.6 links are created.

@eddie-dunn

This comment has been minimized.

Show comment
Hide comment
@eddie-dunn

eddie-dunn Oct 27, 2017

@yyuu I would like to solve this, preferrably in the same place as where the python and python3 symlinks are created. I couldn't find where this is, though.

Could you point me in the right direction?

eddie-dunn commented Oct 27, 2017

@yyuu I would like to solve this, preferrably in the same place as where the python and python3 symlinks are created. I couldn't find where this is, though.

Could you point me in the right direction?

@gangefors

This comment has been minimized.

Show comment
Hide comment
@gangefors

gangefors Oct 30, 2017

Possibly related to #206

gangefors commented Oct 30, 2017

Possibly related to #206

@madhurabhogate

This comment has been minimized.

Show comment
Hide comment
@madhurabhogate

madhurabhogate Nov 29, 2017

@gangefors : The solution you suggested (pyenv virtualenv -p python3.5 3.5.3 myenv) works.
But in my tox.ini I want to verify for multiple python versions say py35, py36. By creating the the virtualenv in above manner; tox runs successfully for py35 version but fails for py36.

ERROR: py36: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\n\nThe `python3.6' command exists in these Python versions:\n 3.6.0\n 3.6.1\n\n"',)

How do we overcome this issues ? Can you suggest please ?

madhurabhogate commented Nov 29, 2017

@gangefors : The solution you suggested (pyenv virtualenv -p python3.5 3.5.3 myenv) works.
But in my tox.ini I want to verify for multiple python versions say py35, py36. By creating the the virtualenv in above manner; tox runs successfully for py35 version but fails for py36.

ERROR: py36: Error creating virtualenv. Note that some special characters (e.g. ':' and unicode symbols) in paths are not supported by virtualenv. Error details: InvocationError('Failed to get version_info for python3.6: b"pyenv: python3.6: command not found\n\nThe `python3.6' command exists in these Python versions:\n 3.6.0\n 3.6.1\n\n"',)

How do we overcome this issues ? Can you suggest please ?

@gangefors

This comment has been minimized.

Show comment
Hide comment
@gangefors

gangefors Nov 29, 2017

@madhurabhogate I would suggest that you create different venvs for different python versions. Then you can use them at the same time with pyenv <shell|local> <py35venv> <py36venv>. The python binaries from both venvs should be available in your path for tox to find.

# Do the following in your shell
$ pyenv virtualenv -ppython3.5 3.5.4 py35
$ pyenv virtualenv -ppython3.6 3.6.3 py36
$ pyenv shell py36 py35

# Then you should have two active environments you can use
$ pyenv versions
  system
  3.5.4
  3.5.4/envs/py35
  3.6.3
  3.6.3/envs/py36
* py35 (set by PYENV_VERSION environment variable)
* py36 (set by PYENV_VERSION environment variable)

# Now you should be able to run tox for both py35 and py36 envlists

gangefors commented Nov 29, 2017

@madhurabhogate I would suggest that you create different venvs for different python versions. Then you can use them at the same time with pyenv <shell|local> <py35venv> <py36venv>. The python binaries from both venvs should be available in your path for tox to find.

# Do the following in your shell
$ pyenv virtualenv -ppython3.5 3.5.4 py35
$ pyenv virtualenv -ppython3.6 3.6.3 py36
$ pyenv shell py36 py35

# Then you should have two active environments you can use
$ pyenv versions
  system
  3.5.4
  3.5.4/envs/py35
  3.6.3
  3.6.3/envs/py36
* py35 (set by PYENV_VERSION environment variable)
* py36 (set by PYENV_VERSION environment variable)

# Now you should be able to run tox for both py35 and py36 envlists
@madhurabhogate

This comment has been minimized.

Show comment
Hide comment
@madhurabhogate

madhurabhogate Nov 29, 2017

@gangefors : Thank you. An example would be really helpful :)

madhurabhogate commented Nov 29, 2017

@gangefors : Thank you. An example would be really helpful :)

@hhstore

This comment has been minimized.

Show comment
Hide comment
@hhstore

hhstore Apr 7, 2018

you can try:

pyenv shell 3.5.3

ref: pyenv/pyenv#856

hhstore commented Apr 7, 2018

you can try:

pyenv shell 3.5.3

ref: pyenv/pyenv#856

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment