proposal: crypto/x509: tighten SSL_CERT_FILE / SSL_CERT_DIR behavior

[FiloSottile]

Currently, on Unix and after #77865 on macOS and Windows, the behavior of $SSL_CERT_FILE / $SSL_CERT_DIR is:

• if the file pointed to by $SSL_CERT_FILE exists, it is loaded
• if the file does not exist, it is silently skipped
• if $SSL_CERT_FILE is empty/unset, the first file from a default list that works is loaded
• if any directories pointed to by $SSL_CERT_DIR exist, they are all loaded
• if they do not exist, they are silently skipped
• if $SSL_CERT_DIR is empty/unset, all the directories from a default list are loaded
• if SetFallbackRoots was/is called, and the logic above produces no roots or x509usefallbackroots=1 is set, the fallback roots are used instead
• on macOS and Windows, if x509sslcertoverrideplatform=0 is set or $SSL_CERT_FILE / $SSL_CERT_DIR are both empty/unset, the platform verifier is used

This is, uh, harder to explain in prose than in code, which is never a good sign.

Also, it has the following unexpected behaviors:

1. even if setting one of $SSL_CERT_FILE / $SSL_CERT_DIR, on Unix the default paths for the other one will be searched
2. if setting $SSL_CERT_FILE / $SSL_CERT_DIR to a path with a typo, no error is surfaced (and the default paths might still get used, or the fallbacks)
3. on macOS and Windows, setting $SSL_CERT_FILE / $SSL_CERT_DIR to an empty file/directory or a path with a typo causes the fallback roots to be used if SetFallbackRoots was called, and an empty pool if SetFallbackRoots was not called (instead of using the platform verifier)

I think we should aim to preserve these invariants instead:

1. if setting either $SSL_CERT_FILE / $SSL_CERT_DIR, that's an absolute statement and we'll either use those or fail
2. the fallback roots are only used if x509usefallbackroots=1 is set, or if no default ones are available on Unix

I propose we change the logic to:

1. if x509usefallbackroots=1 is set and SetFallbackRoots was called, use the fallback roots
2. if x509sslcertoverrideplatform=0 is set on Windows / macOS, use the platform verifier
3. if either/both of $SSL_CERT_FILE / $SSL_CERT_DIR are not empty, they are searched—ignoring the defaults—and if they yield zero roots and an error was encountered, SystemCertPool returns an error (the assumption here is that they might be set to multiple paths to get a fallback behavior, but if none work it was probably unintended, while SSL_CERT_FILE=/dev/null or SSL_CERT_DIR=/var/empty might be intentional)
4. on Unix, search the default paths, and if they yield zero roots, use the fallback roots (if any)
5. on Windows / macOS, use the platform verifier

We should introduce a x509legacyroots=1 GODEBUG setting to revert to the previous behavior, slated to be removed in two years.

/cc @golang/security

[FiloSottile]

One thing worth noting: OpenSSL has the same behavior where setting one of $SSL_CERT_FILE / $SSL_CERT_DIR retains the compile-time defaults for the other. I'd argue it's a misfeature and we should not aim for compatibility.

[gabyhelp]

Related Issues

• proposal: crypto/x509: allow user override of the system roots on all platforms #71924
• crypto/x509: support SSL_CERT_FILE/DIR on Darwin and Windows #77865
• crypto/x509: use platform verifier on Windows, macOS and iOS #46287 (closed)
• proposal: src/crypto/x509: expose paths used by loadSystemRoot publicly #73004 (closed)
• crypto/x509: add standard SSL_CERT_DIR locations #14311 (closed)
• proposal: x509.Certificate.Verify should provide an option for both using the host's root CA set and an additional set of user-provided root CAs #34937 (closed)
• crypto/x509: incompatible behavior of SSL_CERT_DIR with OpenSSL #73961 (closed)

Related Code Changes

• crypto/x509: load certificates from paths in env vars, extra certs dirs
• crypto/x509: load certs from env vars + extra locations
• crypto/x509: rework how system roots are loaded on unix systems

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[tmshort]

One thing worth noting: OpenSSL has the same behavior where setting one of $SSL_CERT_FILE / $SSL_CERT_DIR retains the compile-time defaults for the other. I'd argue it's a misfeature and we should not aim for compatibility.

As an OpenSSL contributor (I don't speak for the OpenSSL organization), I disagree here. Many people rely on that behavior, and having applications (regardless of language) interpret them differently will sow confusion.

It is not uncommon for the these variables to include just the network-local CA, and still depend on the system default CAs when connecting to the rest of the Internet. The proposed solution breaks this paradigm. There is no good method to load the system default CAs in addition to using the variables.

Perhaps a new set of APIs that allow finer grained control:

• x509.SystemCertPool() - returns the system default, ignores the environment variables
• x509.AddSslCertDir() - adds certificates from the given directory
• x509.AddSslCertFile() - adds certificates from the given file
• x509.SetFallbackRoots() - if empty system pool, then use the given pool

Usage:

x509.AddSslCertDir(os.Getenv("SSL_CERT_DIR"))
x509.AddSslCertFile(os.Getenv("SSL_CERT_FILE"))

To maintain identical behavior, all 4 of the above functions would need to be called. Right now, the systempool can't be modified once created (that's a different issue), the above calls would modify the system pool. An appropriate debug variable would be created to maintain existing behavioer once the environment code is removed from x509.SystemCertPool().

Similar calls to modify just a returned pool (i.e. a clone of the systempool or a new CertPool) could be implemented instead/also:

• certpool.AddSslCertDir()
• certpool.AddSslCertFile()

[trivenay]

Related: #80374 addresses the performance aspect of the default-path behavior discussed here.

On Fedora/RHEL/AL2023 systems, the current default path logic loads roots from a bundle file and then redundantly scans the same directory — producing zero new certificates but costing ~820ms on CPU-constrained systems (profiling details in the PR).

The PR defers the default directory scan until a cert verification miss, preserving the existing behavior for user-provided SSL_CERT_DIR paths.