pip install fails in conda create virtual environments (but not in base environment)

[znd4]

Hello!

I am trying to pip install packages into conda virtual environments (i.e. ones created with conda create) and getting

WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProxyError('Cannot connect to proxy.', NewConnectionError('<pip._vendor.urllib3.connection.VerifiedHTTPSConnection object at 0x7f8e1c0482e8>: Failed to establish a new connection: [Errno -2] Name or service not known',))': /simple/black/

(this happens whether I try to install a package from pypi like black, or directly from a git repo via git+ssh (my actual end-goal).

I am able to pip install in the base environment, but I cannot pip install in environments created by conda create. I can also pip install into environments that were created with venv/virtualenv.

I've done a bit of digging into this issue. It usually happens to me when I'm behind a corporate firewall, but I'm not at the moment.

This is the closest thing I've been able to find to the problem I'm running into. The OP said that they resolved the issue by no longer creating virtual environments on remote drives. This makes me think that this might have something to do with the fact that I'm running this on WSL Ubuntu.

Any idea what the issue might be?

Thanks either way :)

p.s.
If I have time, I'll try reproducing this on another windows computer.

Steps to Reproduce

# Running on WSL Ubuntu
# Installation steps
cd ~
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
sh Miniconda3-latest-Linux-x86_64.sh
# Say yes to prompts
# Restart shell to activate base environment
exec $SHELL 
pip install black # should work
# Run in new conda environment
conda create -n py36 python=3.6
conda activate py36
pip install black # Doesn't work
# Run inside of python 3.6 venv
python -m venv ./venv
source ./venv/bin/activate
python -m pip install black # Works

Environment Information

`conda info`

Base `conda info`: ``` active environment : base active env location : /home/zdufour/miniconda3 shell level : 1 user config file : /home/zdufour/.condarc populated config files : conda version : 4.6.14 conda-build version : not installed python version : 3.7.3.final.0 base environment : /home/zdufour/miniconda3 (writable) channel URLs : https://repo.anaconda.com/pkgs/main/linux-64 https://repo.anaconda.com/pkgs/main/noarch https://repo.anaconda.com/pkgs/free/linux-64 https://repo.anaconda.com/pkgs/free/noarch https://repo.anaconda.com/pkgs/r/linux-64 https://repo.anaconda.com/pkgs/r/noarch package cache : /home/zdufour/miniconda3/pkgs /home/zdufour/.conda/pkgs envs directories : /home/zdufour/miniconda3/envs /home/zdufour/.conda/envs platform : linux-64 user-agent : conda/4.6.14 requests/2.21.0 CPython/3.7.3 Linux/4.4.0-17763-Microsoft ubuntu/18.04.1 glibc/2.27 UID:GID : 1000:1000 netrc file : None offline mode : False ```

`conda config --show-sources`

`conda list --show-channel-urls`

``` # packages in environment at /home/zdufour/miniconda3: # # Name Version Build Channel anaconda-client 1.7.2 py37_0 defaults anaconda-project 0.8.2 py37_0 defaults appdirs 1.4.3 pypi_0 pypi asn1crypto 0.24.0 py37_0 defaults astroid 2.2.5 pypi_0 pypi attrs 19.1.0 py37_1 defaults black 19.3b0 pypi_0 pypi ca-certificates 2019.1.23 0 defaults certifi 2019.3.9 py37_0 defaults cffi 1.12.2 py37h2e261b9_1 defaults chardet 3.0.4 py37_1 defaults click 7.0 pypi_0 pypi clyent 1.2.2 py37_1 defaults conda 4.6.14 py37_0 defaults cryptography 2.6.1 py37h1ba5d50_0 defaults decorator 4.4.0 py37_1 defaults filelock 3.0.10 pypi_0 pypi idna 2.8 py37_0 defaults ipython_genutils 0.2.0 py37_0 defaults isort 4.3.20 pypi_0 pypi jsonschema 3.0.1 py37_0 defaults jupyter_core 4.4.0 py37_0 defaults lazy-object-proxy 1.4.1 pypi_0 pypi libedit 3.1.20181209 hc058e9b_0 defaults libffi 3.2.1 hd88cf55_4 defaults libgcc-ng 8.2.0 hdf63c60_1 defaults libstdcxx-ng 8.2.0 hdf63c60_1 defaults mccabe 0.6.1 pypi_0 pypi nbformat 4.4.0 py37_0 defaults ncurses 6.1 he6710b0_1 defaults openssl 1.1.1b h7b6447c_1 defaults pip 19.0.3 py37_0 defaults pluggy 0.11.0 pypi_0 pypi py 1.8.0 pypi_0 pypi pycosat 0.6.3 py37h14c3975_0 defaults pycparser 2.19 py37_0 defaults pylint 2.3.1 pypi_0 pypi pyopenssl 19.0.0 py37_0 defaults pyrsistent 0.14.11 py37h7b6447c_0 defaults pysocks 1.6.8 py37_0 defaults python 3.7.3 h0371630_0 defaults python-dateutil 2.8.0 py37_0 defaults pytz 2019.1 py_0 defaults pyyaml 5.1 py37h7b6447c_0 defaults readline 7.0 h7b6447c_5 defaults requests 2.21.0 py37_0 defaults ruamel_yaml 0.15.46 py37h14c3975_0 defaults setuptools 41.0.0 py37_0 defaults six 1.12.0 py37_0 defaults sqlite 3.27.2 h7b6447c_0 defaults tk 8.6.8 hbc83047_0 defaults toml 0.10.0 pypi_0 pypi tornado 6.0.2 py37h7b6447c_0 defaults tox 3.11.1 pypi_0 pypi tox-conda 0.2.0 pypi_0 pypi traitlets 4.3.2 py37_0 defaults typed-ast 1.3.5 pypi_0 pypi urllib3 1.24.1 py37_0 defaults virtualenv 16.6.0 pypi_0 pypi wheel 0.33.1 py37_0 defaults wrapt 1.11.1 pypi_0 pypi xz 5.2.4 h14c3975_4 defaults yaml 0.1.7 had09818_2 defaults zlib 1.2.11 h7b6447c_3 defaults ```

[bgrosjea]

I had the same problem while specifying a version of python when setting up the virtualenv (working with WSL):

conda create -n myenv python=3.7

This would create a env with python 3.7.7 while my base python is 3.7.6. I also had the problem trying with a lower version (3.5) of python for the virtualenv. I could get pip to install packages only with the same version of python in the base env and the virtualenv.

[songweige]

I have the same issue and was only able to do pip install when the versions are consistent with the python in base env.

[dahvid]

I also have the exact same issue and SongweiGe's solution is the only one that works. After creating a new environment I need to force reinstall the version of pip in the base environment. I'm behind a corporate firewall, if that makes a difference.

[choward232]

Same issue here

[aishvellappally]

Hi,

I faced the same issue when trying to use pip in a newly created conda environment. I am also behind a corporate firewall.I realised that I was using different versions of pip and python in my created environment as compared to the base environment. I tried the solutions provided by @songweige and @bgrosjea and it works , although the solution with Python 3.8.3 and pip 21.0.1 in both base and created conda environment was failing. I finally reinstalled conda with Python 3.8.3 and an earlier version of pip (pip 20.1.1) and applied the same conditions for the created conda environment. Everything works fine now.

[styxjedi]

Hi,

I met the same issue today. And in my case, I finally found that this issue might be caused by different pip.conf config files. One of them is $HOME/.pip/pip.conf, and another is $HOME/.config/pip/pip.conf. I resolved this by deleting $HOME/.config/pip/pip.conf.

[ashvinmanoj]

Same issue. Problem was with the pip version in the Venv. I tried to use the same python version as the base environment still I had the same problem. After downgrading the pip to one version prior, I did not face any warnings.

[dahvid]

Also for me downgrading pip was the solution

…

On Thu, 13 May 2021 at 8:54 ashvinmanoj ***@***.***> wrote: Same issue. Problem was with the pip version in the Venv. I tried to use the same python version as the base environment still I had the same problem. After downgrading the pip to one version prior, I did not face any warnings. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#8691 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWSOM3OF32IA3NILLUVRBTTNNSP5ANCNFSM4HN5PRYQ> .

[zengzhongjie]

The same problem with me in conda env of python 3.7 . I solve this problem by two way:

1. Create a conda env with python 3.6
2. Degrade the pip version to 20.1.1 (orignal is 21.0.1) in base env (python 3.8.3) , and Keep same pip version in conda env(3.7.6)

[PeterQiu0516]

I also have the exact same issue and SongweiGe's solution is the only one that works. After creating a new environment I need to force reinstall the version of pip in the base environment. I'm behind a corporate firewall, if that makes a difference.

I have the same issue and was only able to do pip install when the versions are consistent with the python in base env. And this method works for me.

[wook2014]

Hi,

I faced the same issue when trying to use pip in a newly created conda environment. I am also behind a corporate firewall.I realised that I was using different versions of pip and python in my created environment as compared to the base environment. I tried the solutions provided by @songweige and @bgrosjea and it works , although the solution with Python 3.8.3 and pip 21.0.1 in both base and created conda environment was failing. I finally reinstalled conda with Python 3.8.3 and an earlier version of pip (pip 20.1.1) and applied the same conditions for the created conda environment. Everything works fine now.

Would you please provide where the solution link is? I can't find where it is.

[zhtisi]

I had the same problem... Inspired by this thread, I try to replace the pip of virtual env by pip of base env. And, It works for me.
E.g.,

1. Delete all folders with "pip" prefix in \wsl$\Ubuntu<Your Anaconda Install Location>\anaconda3\envs<Your Venv Name>\lib\python3.7\site-packages
   (This directory is your virtual env's pip)

2. Copy all folders with "pip" prefix in \wsl$\Ubuntu\home\zht\anaconda3\lib\python3.7\site-packages
   (This directory is your base env's pip)

3. Paste into \wsl$\Ubuntu<Your Anaconda Install Location>\anaconda3\envs<Your Venv Name>\lib\python3.7\site-packages

[JPK85]

Same issue here without a corporate firewall. Comes and goes seemingly at random, occasionally works again just by relogging into WSL. I tried creating a symlink from the base environment to a new environment with the same python version but no luck. I also tried zhtisi's solution above with no cigar (still picks up the pip from the environments /bin, not sure if I was supposed to do some cleaning there as well for that to work).

:(

[rsokolewicz]

Same problem here, but not sure if it is related to what is installed in base or having a firewall. What worked for me is creating a new environment with a lower version of python:

conda create --name test python=3.8.10

python=3.8.11, python=3.8.12, and python=3.8.13, all give the same problem as OP. Creating a new environment with 3.9 fails for every version I tried, even though base has python=3.9.12 installed and works fine.

[rsokolewicz]

my problem went away after updating to a new version of anaconda:

conda update -n base -c defaults conda

(which updated conda 4.12.0 to conda 4.13.0 in my case)

[JPK85]

Thanks for the update - I tried updating to conda 4.13.0 per your command but problem unfortunately persists. The issue only occurs for me now with some packages (e.g. pycairo) and works fine with most. Pip installs everything fine until it runs into one of these "problem" packages and after that stops working for anything at all in that environment.

[eamonnkenny]

For me the conda environment worked in WSL Debian for quite some time. Then if you remove it using the correct command "conda env remove -n my_environ" it doesn't work on recreation. However, if I create a new environment with a new name: "conda create -n my_environ2 python=3.9" it mostly doesn't work, however if I create using: "conda create -n my_environ2" pip works perfectly. So, there is something really strange going on. Sometimes it resolves itself a day later and just goes away. I am not using proxy, just a home computer. If it was a proxy issue, or firewall issue, then the second form of environment creation would not succeed (I reckon, could be wrong on this).

I'm convinced its some sort of lease, lock or timeout issue that resolves itself given not running the environment for sometime.

[dahvid]

For me the solution was very firewall specific, I just needed to route both http_proxy and https_proxy to the same proxy server.

…

On Sun, Mar 27, 2022 at 12:00 AM wook2014 ***@***.***> wrote: Hi, I faced the same issue when trying to use pip in a newly created conda environment. I am also behind a corporate firewall.I realised that I was using different versions of pip and python in my created environment as compared to the base environment. I tried the solutions provided by @songweige <https://github.com/SongweiGe> and @bgrosjea <https://github.com/bgrosjea> and it works , although the solution with Python 3.8.3 and pip 21.0.1 in both base and created conda environment was failing. I finally reinstalled conda with Python 3.8.3 and an earlier version of pip (pip 20.1.1) and applied the same conditions for the created conda environment. Everything works fine now. Would you please provide where the solution link is? I can't find where it is. — Reply to this email directly, view it on GitHub <#8691 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABWSOMZSWHGPXVV5F5WUO33VB53ILANCNFSM4HN5PRYQ> . You are receiving this because you commented.Message ID: ***@***.***>

[SiyuanQi]

I met the same problem and solved by installing the same version of python as suggested above.
I found that the specific issue for me is that when resolving recursive dependencies, pip is searching in a different index when the base and new_env use different versions of Python when trying to resolve recursive dependencies.

More details:
When I run pip install foo, it searches for foo in https://mirrors.cloud.aliyuncs.com/pypi/simple/ (which is specified in my pip.conf file) and works. But when it tries to resolve recursive dependencies, it locates the packages in http://mirrors.cloud.aliyuncs.com/pypi/simple/, which uses http instead of https and fails.

[minigeek128]

Hello gys. the acual solution could be very simple. I solved my problem using the conda package manager to install pip inside the virtual env with the command conda install pip and then I installed my packages with pip install and everything worked.