Major problems with current version of pyenv-virtualenv

[fgimian]

Hey there Yuu, hope you're well 😄

Since some of the latest commits, I've been having quite a few major problems with this extension I'm afraid.

My test system is a fresh install of Ubuntu Server 12.04 with the regular bash shell.

This output should explain the problems 😄

fots@fotsies-ubprecise-02:~$ pyenv versions
* system (set by /home/fots/.pyenv/version)
  2.7.7
  3.4.1
fots@fotsies-ubprecise-02:~$ python -V
Python 2.7.3
fots@fotsies-ubprecise-02:~$ pyenv global 2.7.7
fots@fotsies-ubprecise-02:~$ python -V
Python 2.7.7
fots@fotsies-ubprecise-02:~$ pyenv virtualenv newproject
New python executable in /home/fots/.pyenv/versions/newproject/bin/python
Installing setuptools, pip...done.
fots@fotsies-ubprecise-02:~$ pyenv activate newproject
(newproject)fots@fotsies-ubprecise-02:~$ python -V
Python 2.7.3
(newproject)fots@fotsies-ubprecise-02:~$ pyenv deactivate
(newproject)fots@fotsies-ubprecise-02:~$ unset PYENV_VERSION
fots@fotsies-ubprecise-02:~$ pyenv virtualenv 2.7.7 newproject2
New python executable in /home/fots/.pyenv/versions/newproject2/bin/python
Installing setuptools, pip...done.
fots@fotsies-ubprecise-02:~$ pyenv activate newproject2
(newproject2)fots@fotsies-ubprecise-02:~$ python -V
Python 2.7.3
(newproject2)fots@fotsies-ubprecise-02:~$

So in summary:

• Creating a virtualenv always uses the system version of Python
• Deactivating a virtualenv no longer correctly unsets the PYENV_VERSION environment variable

Your help would be greatly appreciated
Fotis

[yyuu]

From v20140614, the auto-activation feature (#24) has been added in pyenv-virtualenv. If the eval "$(pyenv virtualenv-init -)" has been called in the user's shell as well as eval "$(pyenv init -)", it will call activate and deactivate automatically based on the version in pyenv.

Unfortunately, this feature requires to not set pyenv's shell version during pyenv activate and pyenv deactivate. I would recommend you to try these options:

1. Try setting eval "$(pyenv virtualenv-init -)"
2. Call pyenv shell manually
   1. Call pyenv shell venv before calling pyenv activate
   2. Call pyenv shell --unset after calling pyenv deactivate

I am sorry for this incompatible change, but I believe that the auto-activation feature is valuable despite the incompatibility.

[fgimian]

Hey there Yuu, thanks for your quick reply.

I now understand the auto-activation feature and it is indeed great. I suggest you disable the pyenv activate and pyenv deactivate commands if the user has added eval "$(pyenv virtualenv-init -)" to their profile, or present a warning if the user uses them to explain that they must disable virtualenv-init to use them.

This explains and resolves my second issue, but any ideas about the first one?

Every virtualenv I create uses the system Python:

e.g.

fots@fotsies-ubprecise-02:~$ pyenv versions
* system (set by /home/fots/.pyenv/version)
  2.7.7
fots@fotsies-ubprecise-02:~$ pyenv global 2.7.7
fots@fotsies-ubprecise-02:~$ mkdir myproject
fots@fotsies-ubprecise-02:~$ cd myproject/
fots@fotsies-ubprecise-02:~/myproject$ python -V
Python 2.7.7
fots@fotsies-ubprecise-02:~/myproject$ pyenv virtualenv myproject
New python executable in /home/fots/.pyenv/versions/myproject/bin/python
Installing setuptools, pip...done.
fots@fotsies-ubprecise-02:~/myproject$ pyenv local myproject
(myproject)fots@fotsies-ubprecise-02:~/myproject$ python -V
Python 2.7.3
(myproject)fots@fotsies-ubprecise-02:~/myproject$ cd ..
fots@fotsies-ubprecise-02:~$ python -V
Python 2.7.7

I would really appreciate your help with this as I really rely on and love pyenv 😄

Cheers
Fotis

[yyuu]

I tried same but could not reproduce on my Ubuntu 12.04 :(

[yamashita@lima1:~]% pyenv versions
* system (set by /home/yamashita/.pyenv/version)
  2.7.7
[yamashita@lima1:~]% pyenv global 2.7.7
[yamashita@lima1:~]% mkdir myproject
[yamashita@lima1:~]% cd myproject
[yamashita@lima1:~/myproject]% python -V
Python 2.7.7
[yamashita@lima1:~/myproject]% pyenv virtualenv myproject
New python executable in /home/yamashita/.pyenv/versions/myproject/bin/python
Installing setuptools, pip...done.
[yamashita@lima1:~/myproject]% pyenv local myproject
[yamashita@lima1:~/myproject]% python -V
Python 2.7.7
[yamashita@lima1:~/myproject]% cd ..
[yamashita@lima1:~]% python -V
Python 2.7.7

[yyuu]

Actually, pyenv virtualenv-init uses pyenv activate and pyenv deactivate internally. They cannot be disabled even if the pyenv virtualenv-init has configured.

And, I have already setup the notice for setup instruction around pyenv virtualenv-init in pyenv activate. Now pyenv activate displays the instruction if the pyenv virtualenv-init is not configured in user's shell profile.

https://github.com/yyuu/pyenv-virtualenv/blob/4a6ef1134978859dbda78a19d5cd8640a0b0def4/bin/pyenv-sh-activate#L62

[fgimian]

I think I have found the trigger for the problem with Python version when using pyenv-virtualenv. If a virtualenv binary is available in the path before you install pyenv, pyenv seems to use that binary all the time.

Here's an exact step by step which I followed to reproduce the problem on a completely fresh install of Ubuntu 12.04:

sudo apt-get update

# Install build requirements
sudo apt-get install curl git-core gcc make zlib1g-dev libbz2-dev libreadline-dev libsqlite3-dev libssl-dev

# Install virtualenv on the system Python
sudo apt-get install python-virtualenv

# Install and activate pyenv
curl -L https://raw.github.com/yyuu/pyenv-installer/master/bin/pyenv-installer | bash
echo 'export PATH="$HOME/.pyenv/bin:$PATH"' >> .profile
echo 'eval "$(pyenv init -)"' >> .profile
echo 'eval "$(pyenv virtualenv-init -)"' >> .profile
source .profile

# Install Python 2.7.7 and create a new virtualenv using it as the Python version
pyenv install 2.7.7
pyenv virtualenv 2.7.7 myvenv
mkdir project
cd project
pyenv local myvenv

# Print the Python version (I see 2.7.3)
python -V

From this point, we can clearly see what's going on here now, pyenv seems to be using the system virtualenv binary:

fots@fotsies-ubprecise-template:~$ pyenv global 2.7.7
fots@fotsies-ubprecise-template:~$ pyenv version
2.7.7 (set by /home/fots/.pyenv/version)
fots@fotsies-ubprecise-template:~$ pyenv which python
/home/fots/.pyenv/versions/2.7.7/bin/python
fots@fotsies-ubprecise-template:~$ pyenv which virtualenv
/usr/bin/virtualenv
fots@fotsies-ubprecise-template:~$ pyenv which pip
/home/fots/.pyenv/versions/2.7.7/bin/pip
fots@fotsies-ubprecise-template:~$ pyenv which easy_install
/home/fots/.pyenv/versions/2.7.7/bin/easy_install

Cheers
Fotis

[yyuu]

Thanks for the detailed procedure!!

The python-virtualenv package might causes a problem if you have installed
pyenv with pyenv-which-ext plugin which is installed via pyenv-installer.
In this case, doing 'pip install virtualenv' on 2.7.7 by hand might be a
possible workaround.

I'll try to reproduce and fix this. Thanks again.

[fgimian]

Awesome, thank you so much! Can you also re-consider this commit too please 8a83775 This is where the problem with virtualenv deactivation was introduced.

It would be ideal if the old behaviour of pyenv-virtualenv could be maintained for those who do not wish to add eval "$(pyenv virtualenv-init -)". The fact that deactivate doesn't work and properly unset PYENV_VERSION is quite difficult to work with to be honest. It forces users to the new methodology which is great in some circumstances but not necessarily others.

Thanks again for all your help and keep up the great work!! 😄

Cheers
Fotis

[yyuu]

@fgimian I pushed d9e3adc as a workaround for incompatibility of pyenv activate and pyenv deactivate. With that fix, activate/deactivate will just work as same as before v20140614. Please try it if it is working as expectedly.

[fgimian]

Thank you so much. I've just tested out the newest version but it still seems to leave PYENV_VERSION set after deactivating a virtualenv. Any ideas?

To clarify, I've only added the following to my .profile:

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"

The prompt does change when I de-activate the virtualenv but the environment variable stays set. This causes a number of issues later on and I'm forced to unset it manually.

The line which was needed to have this work was removed in the commit I linked above. The line I'm referring to is:

echo "pyenv shell --unset"

From the bin/pyenv-sh-deactivate file.

Sorry for all the trouble 😄
Fotis

[yyuu]

It sounds strange... In d9e3adc, bin/pyenv-sh-deactivate calls pyenv shell --unset if PYENV_VIRTUALENV_INIT is empty (== pyenv virtualenv-init is not configured). Please make sure you don't have environment variable of PYENV_VIRTUALENV_INIT.

d9e3adc#diff-d20ab819f2abd9256d4859bfd1453b56R18

[fgimian]

Thanks again for your help, this is rather strange as I don't have PYENV_VIRTUALENV_INIT set.

Please see the output below:

fots@fotsies-ubprecise-01:~$ env | egrep PYENV
PYENV_SHELL=bash
fots@fotsies-ubprecise-01:~$ tail -5 ~/.profile

export PATH="$HOME/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
# eval "$(pyenv virtualenv-init -)"

fots@fotsies-ubprecise-01:~$ pyenv virtualenv test
New python executable in /home/fots/.pyenv/versions/test/bin/python
Installing setuptools, pip...done.
fots@fotsies-ubprecise-01:~$ pyenv activate test
(test)fots@fotsies-ubprecise-01:~$ env | egrep PYENV
PYENV_VERSION=test
PYENV_SHELL=bash
(test)fots@fotsies-ubprecise-01:~$ pyenv deactivate
fots@fotsies-ubprecise-01:~$ env | egrep PYENV
PYENV_VERSION=test
PYENV_SHELL=bash

Notice that PYENV_VERSION is not unset when I deactivate the virtualenv?

Cheers
Fotis

[yyuu]

Thanks for your help too. Please let me know the output from 'env
PYENV_DEBUG=1 pyenv sh-deactivate'. Because it might be quite long, please
use gist to upload.

[fgimian]

My pleasure, here's the gist for you https://gist.github.com/fgimian/d121383b038d74c97859

Hope this helps and so sorry for the trouble
Fotis

[yyuu]

Thanks! I looked the gist, and I found a line echo 'declare -f deactivate 1>/dev/null 2>&1 && deactivate'.

https://gist.github.com/fgimian/d121383b038d74c97859#file-gistfile1-txt-L1164

In d9e3adc, the line should be declare -f deactivate 1>/dev/null 2>&1 && deactivate; (notice at trailing ';').

https://github.com/yyuu/pyenv-virtualenv/blob/d9e3adc1085d5f8ddc525088bf8532a3c080f971/bin/pyenv-sh-deactivate#L15

I think you are working with wrong revision. Please try again with updating pyenv-virtualenv at d9e3adc.

$ cd "$(pyenv root)/plugins/pyenv-virtualenv"
$ git fetch
$ git checkout issue26
$ git reset --hard d9e3adc1085d5f8ddc525088bf8532a3c080f971

[fgimian]

Aah that works perfectly. My apologies, I thought the fix was in master 😄

Thanks so much! Would you consider merging this fix on the next release?

Cheers
Fotis

[yyuu]

Yep, I'll merge and release it soon.

[fgimian]

Awesome, thanks for everything and keep up the great work!! 😄