Respect SSL_CERT_FILE environment variable

[thetredev]

I want to reiterate on the closed issue #200. I understand the following point being made by @davisagli, but I don't agree with it:

The whole point of certifi is to provide a specific bundle of CA certificates. If you want to use a different bundle, configure an ssl.SSLContext to load that bundle instead of using certifi.

However, the point following up to the previous one by @alex just isn't true, at least for certifi:

Yes, python already respects the SSL_CERT_FILE environment variable.

While setting SSL_CERT_FILE might work with Python in general, it doesn't work at all when using certifi:

# my self-hosted GitLab uses a self-signed certificate signed by my own custom CA
$ export MY_GITLAB_HOST="gitlab.mydomain.lan"

# OpenSSL has no problem with it
$ echo "" | openssl s_client -connect ${MY_GITLAB_HOST}:443 -servername ${MY_GITLAB_HOST} 2>&1 | grep -e 'Verification:\s*OK'
Verification: OK

# let's see if certifi works as well
$ pip freeze | grep certifi
certifi==2024.2.2

# set SSL_CERT_FILE to a valid value as mentioned in issue #200
$ export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt

# test certifi and realize it doesn't work
$ python3 test-certifi.py
Traceback (most recent call last):
  File "/home/build/test-certifi.py", line 18, in <module>
    verify_ssl_certificate(hostname_to_check)
  File "/home/build/test-certifi.py", line 12, in verify_ssl_certificate
    with context.wrap_socket(sock, server_hostname=hostname) as ssock:
  File "/usr/lib/python3.10/ssl.py", line 513, in wrap_socket
    return self.sslsocket_class._create(
  File "/usr/lib/python3.10/ssl.py", line 1100, in _create
    self.do_handshake()
  File "/usr/lib/python3.10/ssl.py", line 1371, in do_handshake
    self._sslobj.do_handshake()
ssl.SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self-signed certificate in certificate chain (_ssl.c:1007)

What's test-certifi.py?

# copied and modified from https://www.askpython.com/python/python-program-to-verify-ssl-certificates

import certifi
import os
import ssl
import socket

def verify_ssl_certificate(hostname):
    context = ssl.create_default_context(cafile=certifi.where())
 
    with socket.create_connection((hostname, 443)) as sock:
        with context.wrap_socket(sock, server_hostname=hostname) as ssock:
            ssock.do_handshake()
            cert = ssock.getpeercert()
            print("Certificate is valid.")
 
hostname_to_check = os.getenv("MY_GITLAB_HOST")
verify_ssl_certificate(hostname_to_check)

I don't want to modify certifi's cacert.pem file just to make other Python tools/libraries happy which just so happen to use certifi and don't agree with OpenSSL. I don't see a reason why SSL_CERT_FILE should not be supported. Could you please reiterate and elaborate more on this? A simple check for SSL_CERT_FILE and returning that path, if the value is set, would be the simplest fix to my, and apparently many other people's problems with self-signed certificates and Python stuff that's using certifi for good reason.

Thank you!

[sigmavirus24]

You are explicitly specifying a TLS bundle with certifi.where() in your SSLContext. What do you expect?

No where in the issue you linked does it say that certifi supports that. Python does, unless you tell it to ignore it. No reasonable tool or library respects an environment variable when the software has been given an explicit value. It will fallback to the environment.

And as explained in that issue, this library provides a specific known good set of roots to trust. There's no need to completely baffle users by allowing an environment variable to change that behaviour, when users can do the right thing in the first place.

[thetredev]

Yes, I do explicitly specify the context using certifi.where(), as an example. If a third party library I install via PyPi uses that context, I have no control over which CA it should include. This is my point.

[sigmavirus24]

That's poor design by that library, that isn't this library's responsibility

[sethmlarson]

I'm in agreement that certifi is doing the right thing here. The real solution to user-configured certificates is to use system certificates (see the "truststore" project), not expecting certifi to change its behavior.