miniconda3 py2.7 env picks up packages outside env

[sarah-peter]

Current Behavior

When I create a conda environment with python 2.7, the python installation picks up packages that I installed with the system python 2.7 before using conda (e.g. with pip install --user in my home directory).

Steps to Reproduce

$ conda create -n test python=2.7
$ conda activate test
$ python
Python 2.7.16 |Anaconda, Inc.| (default, Mar 14 2019, 21:00:58)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path
['', '/home/users/sdiehl/tools/miniconda3/envs/test/lib/python27.zip', '/home/users/sdiehl/tools/miniconda3/envs/test/lib/python2.7', '/home/users/sdiehl/tools/miniconda3/envs/test/lib/python2.7/plat-linux2', '/home/users/sdiehl/tools/miniconda3/envs/test/lib/python2.7/lib-tk', '/home/users/sdiehl/tools/miniconda3/envs/test/lib/python2.7/lib-old', '/home/users/sdiehl/tools/miniconda3/envs/test/lib/python2.7/lib-dynload', '/home/users/sdiehl/.local/lib/python2.7/site-packages', '/home/users/sdiehl/tools/miniconda3/envs/test/lib/python2.7/site-packages']

Note the /home/users/sdiehl/.local/lib/python2.7/site-packages path which should not be there.

Expected Behavior

The purpose of conda should be to isolate the environment from any previously installed packages either in the user's home or system-wide.

Environment Information

`conda info`

     active environment : test
    active env location : /home/users/sdiehl/tools/miniconda3/envs/test
            shell level : 2
       user config file : /home/users/sdiehl/.condarc
 populated config files :
          conda version : 4.6.14
    conda-build version : not installed
         python version : 3.7.3.final.0
       base environment : /home/users/sdiehl/tools/miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/users/sdiehl/tools/miniconda3/pkgs
                          /home/users/sdiehl/.conda/pkgs
       envs directories : /home/users/sdiehl/tools/miniconda3/envs
                          /home/users/sdiehl/.conda/envs
               platform : linux-64
             user-agent : conda/4.6.14 requests/2.21.0 CPython/3.7.3 Linux/3.10.0-693.21.1.el7.x86_64 centos/7.6.1810 glibc/2.17
                UID:GID : 5426:666
             netrc file : None
           offline mode : False

`conda config --show-sources`

`conda list --show-channel-urls`

# packages in environment at /home/users/sdiehl/tools/miniconda3/envs/test:
#
# Name                    Version                   Build  Channel
ca-certificates           2019.5.15                     0    defaults
certifi                   2019.3.9                 py27_0    defaults
libedit                   3.1.20181209         hc058e9b_0    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc-ng                 9.1.0                hdf63c60_0    defaults
libstdcxx-ng              9.1.0                hdf63c60_0    defaults
ncurses                   6.1                  he6710b0_1    defaults
openssl                   1.1.1c               h7b6447c_1    defaults
pip                       19.1.1                   py27_0    defaults
python                    2.7.16               h9bab390_0    defaults
readline                  7.0                  h7b6447c_5    defaults
setuptools                41.0.1                   py27_0    defaults
sqlite                    3.28.0               h7b6447c_0    defaults
tk                        8.6.8                hbc83047_0    defaults
wheel                     0.33.4                   py27_0    defaults
zlib                      1.2.11               h7b6447c_3    defaults

[sarah-peter]

After some help I figured out that this is a python setting and can be disabled with export PYTHONNOUSERSITE=True. There are already a bunch of issues related to this: #6018, #3162, #448, #394 and #7707, so I wil close this one.

[mingwandroid]

The purpose of conda should be to isolate the environment from any previously installed packages either in the user's home or system-wide

IMHO this is not what the purpose of conda should be.

[msarahan]

That is a use case, yes. It is not conda's purpose, though. I have been bitten badly by this issue many times before. I don't like this feature of python. Still, it is a feature, and we can't capriciously remove features from upstream.

[sarah-peter]

@mingwandroid @msarahan I understood that you don't want to change this behaviour, that's why I closed the ticket. However from the amount of tickets related to this it seems that many people expect a different behaviour.

You should also be aware that this causes unwanted side effects. In my case for example, I told conda explicitly to install numpy 1.16.4 and conda list also shows it, but after closer inspection it never installed any numpy at all and instead uses a much older version which I had in my home directory.

[msarahan]

Oh yes, we understand. We find this behavior unintuitive as well, but you have to understand that this isn't our decision. The greater Python community specified this behavior in pep 370: https://www.python.org/dev/peps/pep-0370/

If we get rid of that, we are seen as a non standard python distribution and get flak for it.

What you describe as a side effect is the intent of pep370. Conda is correctly installing numpy for you, but the version you already have in your user site packages folder gets found first. Remember, conda is not the same thing as python. Just because a conda package is installed doesn't mean python won't find a different thing earlier in its search path. We could perhaps make a package, say "safe-mode" that would set all the env vars such that python would strictly use only software from the conda env, but we can't disable pep370 behavior by default.

[sarah-peter]

@msarahan I'm pretty sure that conda did NOT correctly install numpy in the specified version. It was not in the site-packages directory of the environment. In sys.path the environment comes before my home folder and still it takes the numpy from my home folder, which means it didn't find numpy in the environment folders.

[msarahan]

That's fun. If it's missing from the env site packages dir, that's a bigger problem. Can you zip up the conda-meta folder from your env and upload it here?

[sarah-peter]

All the testing I did must have confused me. Actually it is the other way around. Conda installed numpy in the proper version, but in sys.path my home directory site-packages is listed before the env site-packages.
My apologies for the confusion.

[msarahan]

No problem. Always glad when there's not a terrible conda issue to try to figure out.

[ackalker]

@msarahan I beg to disagree that implementing in conda a way to disable Python adding user site to sys.path would go against PEP 370, in fact both the text of PEP 370 and the official Python documentation on PYTHONNOUSERSITE (which actually refers to PEP 370) state that setting this environment variable can be used for exactly this purpose.
I hope that you will reconsider implementing such a feature in conda.

Meanwhile I would like to provide a workaround which works for me, using Bash under Linux, and with conda configured not to activate the base environment automatically (this is important!) using:

$ conda config --set auto_activate_base false

Create an alias for conda, place the following line in ~/.bashrc:

alias conda='export PYTHONNOUSERSITE=1; conda'

This will set and export the PYTHONNOUSERSITE environment variable whenever any conda command is run, the first of which will probably be something like conda activate <some-environment>.

To verify that the workaround is working, run:

$ exec $SHELL
$ conda activate
$ python -m site

which on my system outputs:

sys.path = [
    '/home/miki',
    '/home/miki/anaconda3/lib/python38.zip',
    '/home/miki/anaconda3/lib/python3.8',
    '/home/miki/anaconda3/lib/python3.8/lib-dynload',
    '/home/miki/anaconda3/lib/python3.8/site-packages',
]
USER_BASE: '/home/miki/.local' (exists)
USER_SITE: '/home/miki/.local/lib/python3.8/site-packages' (exists)
ENABLE_USER_SITE: False

[mingwandroid]

and the official Python documentation on PYTHONNOUSERSITE (which actually refers to PEP 370) state that setting this environment variable can be used for exactly this purpose.

Yes, and you should set it whenever you need it. I don't believe that conda should do anything like that though.

Your Alias approach is what I would recommend but I would call it something other than conda! Something like conda-nolocal

[ackalker]

Thanks for the tip, I will rename the alias.

[DanielGoldfarb]

Yes, and you should set it whenever you need it. I don't believe that conda should do anything like that though.

I'm not convinced that conda should not do this. I understand the logic to say that conda should not mess with "standard" python behavior, but there is an argument to say that standard python behave is, in fact, otherwise:

Admittedly, I am relatively new to conda: I tend to think of conda environments somewhat similar to python virtual environments, at least conda environments other than "(base)". Please correct me if this is incorrect thinking (as my entire argument below is based on this view).

When I create a python virtual environment, it uses python from that virtual environment, and similarly it only uses python packages installed in that virtual environment.

When I activate a conda environment, it clearly uses the version of python installed under miniconda and not the python that was installed on my system prior to my installing miniconda. Therefore I would expect that it would also only use the python packages installed (at least) under miniconda (if not only those installed in the particular conda environment).

That's my argument for this. At the very least it would be nice if this was part of the conda configuration, so that one can configure whether a conda environment will include previously installed python and/or python packages. (And, imho, the default configuration should be that a conda environment is isolated).

The alias workaround is a good workaround! (Thanks Alain). However, personally my preference is to avoid aliasing commands, unless absolutely necessary, since it is not uncommon for me, from day to day, to find myself working on a number of different machines (owned by various organizations) with various environments, and then I have to remember that something may not be working because a command is aliased on one machine but not another.

That's my two cents. You get what you pay for ;-) Thoughts? Comments?
Thanks.

[github-actions]

Hi there, thank you for your contribution to Conda!

This issue has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue if needed.