curl: add --ca-native and --proxy-ca-native

[bagder]

These are two boolean options to ask curl to use the native OS's CA store when verifying TLS servers. For peers and for proxies respectively.

They currently only have an effect for curl on Windows when built to use OpenSSL for TLS.

[dfandrich]

What are the intended semantics on non-Windows systems? A user might expect, for examples, that selecting this on macOS would lead to switching to the SecureTransport back-end. Or on other systems switching to OpenSSL. Doesn't NSS have its own certificate store that this option might be expected to affect?

[bagder]

What are the intended semantics on non-Windows systems?

If/when someone adds support for non-Windows systems I figure they could work the same?

Doesn't NSS have its own certificate store that this option might be expected to affect?

I don't think so. It is still not the OS's store.

[noloader]

I think the other case to consider, in contrast to @dfandrich comment, is what happens when OpenSSL is configured to use the Windows Certificate store, but cURL was not invoked with --ca-native or --proxy-ca-native. In this case the user may get the Windows Certificate store even though they did not want it. They may have wanted to use cURL's cacerts.pem.

I haven't looked at this area of OpenSSL in several years. My apologies if I got my wires crossed.

[bagder]

what happens when OpenSSL is configured to use the Windows Certificate store, but cURL was not invoked with --ca-native or --proxy-ca-native

OpenSSL is never configured to do this by itself. It does not offer this feature.

This is code in libcurl that we activate with an curl_easy_setopt() option. Thus, if none of the *ca-native options is used with the tool, OpenSSL will use the CA store provided by other options.

[noloader]

OpenSSL is never configured to do this by itself. It does not offer this feature.

Hmmm... I think enable-capieng tells OpenSSL to use the Windows Certificate store. See https://wiki.openssl.org/index.php/Compilation_and_Installation#Configure_Options.

Do you know if that option was removed?

[bagder]

OpenSSL is never configured to do this by itself. It does not offer this feature.

Hmmm... I think enable-capieng tells OpenSSL to use the Windows Certificate store. See https://wiki.openssl.org/index.php/Compilation_and_Installation#Configure_Options.

Do you know if that option was removed?

I don't know what that is, but I know that we don't use that.

[vszakats]

capieng is an OpenSSL / quictls build option.

curl-for-win has it disabled, because by default it uses external engine DLLs, which in turn needs to have dynamic load capability enabled, which in turn still seems insecure in OpenSSL due to its incomplete fix for a 2019 CVE. Due to this, I disabled capieng with the other engines, about a year ago.

I was tracking this PR for Windows CA certificate store support. It landed 10 days ago in quictls 3.1.0. It seems to implement support internally and it's part of the curl.exe released the same day.

What is necessary now in OpenSSL to make use of this feature from curl? Do we still need capieng for it?

[jay]

Basically I don't know but:

It looks like according to their CHANGES for OpenSSL 3.1 they've added SSL_CERT_URI and for OpenSSL 3.2 they've added SSL_CERT_PATH and are going to deprecate SSL_CERT_DIR (which could do both but had some parsing problem, still I don't get why they would deprecate it for paths). Both commits openssl/openssl@606e0426 and openssl/openssl@021859bf (also its parent) are in master and not in openssl-3.1 branch.

curl tool reads SSL_CERT_DIR (note it's ignored for Schannel) and sets that as the path. I don't know if OpenSSL is now reading the environment itself but the URI is org.openssl.winstore:// not capieng. If you have a master build then try SSL_CERT_URI=org.openssl.winstore:// curl ... and if that doesn't work try curl --capath "org.openssl.winstore://" ...

[vszakats]

So it seems capieng is not playing a role here. And the OpenSSL patches mentioned did not end up in 3.1, and are now targeted for 3.2.

[jay]

I really don't know. That was my impression. You may want to ask in the thread.

[vszakats]

capieng is a legacy component that didn't see major feature updates in the recent years, and OpenSSL is/was well-known for not supporting the Windows CA store natively. This tells that capieng is not involved. The mentioned PR originally listed this feature for 3.1 in CHANGES, but it's missing from the release. Strings SSL_CERT_URI and winstore are also missing from the source tarball. SSL_CERT_DIR and SSL_CERT_FILE are present, but CertOpenSystemStore is not.

None of the popular forks support this either. wolfSSL does however, seemingly also when using its OpenSSL-compatibility API, though I haven't tested it.