use venv with pyenv in company network with it's own CA

[john-soda]

Im working in company network with it's own CA.
When I want to Download any pip packages I get always ssl certificate_verify_failed
Therefore I use export REQUESTS_CA_BUNDLE=/etc/ssl/certs/Web_Gateway_CA.pem
What would be the best and correct way to handle this problem without exporting it everytime after I activate the venv?

[joshfriend]

pyenv is a tool for installing python, whatever issues you have with corporate networks and Pip is not something pyenv can help you with.

I'd put REQUESTS_CA_BUNDLE in your .bashrc or equivalent so you dont have to export it for every new shell.

[john-soda]

The reason why I'm asking here this question is because my normal system installation pip and in every venv created without pyenv pip is using the system certificates and I don't have this problem. So I have this problem exclusively when I use pyenv. I hoped to find here some information which brings me closer to a solution.

[SpringbokBE]

I am facing the very same problem and have to use the same workaround. I guess it is related to the fact that the pyenv Python versions are being installed in the ~/.pyenv/versions folder rather than in a system-wide folder such as /usr/local. Not sure whether it is possible to install pyenv Python versions system-wide, though.

[native-api]

The reason why I'm asking here this question is because my normal system installation pip and in every venv created without pyenv pip is using the system certificates and I don't have this problem.

"System installation Pip" is patched by distribution's provider to achieve this effect. (E.g. for Ubuntu, see the patches subdirectory in the non-original tarball at https://packages.ubuntu.com/focal-updates/python3-pip to see what patches they apply). To do the same in Pyenv, we'd need to know and mimic the relevant changes they make somehow.

[native-api]

Therefore I use export REQUESTS_CA_BUNDLE=/etc/ssl/certs/Web_Gateway_CA.pem

Specifically for your case, it should help if you add that certificate into the system's certificate bundle as per https://askubuntu.com/questions/73287/how-do-i-install-a-root-certificate
(Tbh your sysadmin should have done that for you -- unless they e.g. only want a few select programs to use that CA.)

(If that doesn't help, we'd need more info on how exactly your sysadmin has configured the system to use the local CA to be able to say anything further.)

[bcjarrett]

Hello - I am having the same issue reported above. I am the sysadmin and a python user.

We're running Ubuntu 20.04. I have installed our trusted root CA to the system. Command line tools, like curl or wget or the system pip all work as expected. Where as the .pyenv/shims/pip does not. e.g.

pip install fastapi
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1123)'))': /simple/fastapi/

To get over the issue temporarily, we're adding --trusted-host pypi.org --trusted-host files.pythonhosted.org to our pip installs.

I'll need to revisit this in the future, but have added it to our backlog. If you have any thoughts or if I can help by providing logs of some sort let me know. I'll update this thread if I identify a better solution.

[native-api]

For starters, you can set any Pip arguments via appropriately-named envvars.
Then, see HTTPS Certificates - pip documentation. According to that, by default, Pip uses the certificate store from the bundled certifi module.

[bcjarrett]

It's worthwhile to add here that the python binary is also referencing some CA store other than the system one. So these SSL issues are present with pip and in the IDLE environment:

>>> request = urllib.request.Request("https://exmaple.com/")
>>> req = urllib.request.urlopen(request)
Traceback (most recent call last):
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/urllib/request.py", line 1346, in do_open
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/http/client.py", line 1285, in request
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/http/client.py", line 1331, in _send_request
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/http/client.py", line 1280, in endheaders
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/http/client.py", line 1040, in _send_output
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/http/client.py", line 980, in send
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/http/client.py", line 1454, in connect
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/ssl.py", line 500, in wrap_socket
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/ssl.py", line 1040, in _create
  File "/root/.pyenv/versions/3.9.12/lib/python3.9/ssl.py", line 1309, in do_handshake
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1129)

Again, no issues when using the system interpreter.

[native-api]

@bcjarrett #2257