curl: support embedding a CA bundle

[vszakats]

Add the ability to embed a CA bundle into the curl binary. It is used
when no other runtime or build-time option set one.

This helps curl-for-win macOS and Linux builds to run standalone, and
also helps Windows builds to avoid picking up the CA bundle from an
arbitrary (possibly world-writable) location (though this behaviour is
not currently disablable).

Usage:

• cmake: -DCURL_CA_EMBED=/path/to/curl-ca-bundle.crt
• autotools: --with-ca-embed=/path/to/curl-ca-bundle.crt
• Makefile.mk: CURL_CA_EMBED=/path/to/curl-ca-bundle.crt

Also add new command-line option --dump-ca-embed to dump the embedded
CA bundle to standard output.

Closes #14059

---

TODO:

• rebase on cmake: sync CA bundle/path detection with autotools #14182
• skip embedding if no TLS backend support it (both autotools and cmake)

Ideas:

• gzipped embedding? (230KB → 130KB)
  with reusable unzipper function inside src/tool_something.c
• move hugehelp to use that too? (ATM the unzip code is part of the generated source)
• move up cacert embedding to libcurl?
• strip CR bytes before embedding for better reproducibility?
• generate the dummy tool_ca_embed.c also in CMake/Makefile.mk to keep binary output in sync? (though the dummy code is likely discarded by the linker in autotools builds.)
• build-time option to disable FindWin32CACert() feature? → curl: add options for safe/no CA bundle search (Windows) #14582

[bagder]

It might make sense to display this as a feature in curl -V so that a user can tell if it has a cacert bundle built-in or not.

[bagder]

Do we want to fall back to the old malloc-less feature display in case malloc() fails?

When malloc fails (which is basically not possible on most modern systems) we should bail out. Not try something else. IMHO.

[vszakats]

Thanks, I agree, leaving it as-is.

On another question: Do we want a --dump-embedded-cacert option (which would write the embedded CA bundle to stdout and quit)? Any preference for the option name?

It'd be useful for transparency, i.e. to see the trusted roots without peeking into the binary with a capable viewer (or strings). → added as --dump-ca-embed.

[bagder]

Do we want a --dump-embedded-cacert option (which would write the embedded CA bundle to stdout and quit)? Any preference for the option name?

I think such a command line option makes a lot of sense, yes. I don't have any better name suggestion!

[vszakats]

I went with --dump-ca-embed. Rhymes with the rest of new symbols added. Not too long.

[vszakats]

Patch that allows disabling the Windows-specific PATH-search for a CA bundle:

--- a/src/tool_doswin.c
+++ b/src/tool_doswin.c
@@ -621,7 +621,7 @@ CURLcode FindWin32CACert(struct OperationConfig *config,
 {
   CURLcode result = CURLE_OK;
 
-#ifdef CURL_WINDOWS_APP
+#if defined(CURL_WINDOWS_APP) || defined(CURL_WINDOWS_SKIP_SEARCHPATH_CACERT)
   (void)config;
   (void)backend;
   (void)bundle_file;

This option becomes useful when combined with embedded bundles, as it prevents curl from picking one up from a "random" disk location, including world-writable ones. It still allows to specify one manually either via the documented envs or command-line / .curlrc options, overriding the embedded default.

Does this make sense? Does it raise any concern?

[vszakats]

Embedded certs is now tested via curl-for-win. Also in this PR, after this commit and re-running the workflow: curl/curl-for-win@b4945dd

[bagder]

gzipped embedding? (230KB → 130KB)

A downside with compressing this:

Since the TLS libraries cannot use the blob compressed, this would then have to use 230KB allocated memory to store the cacert bundle in, in addition to the 130KB compressed read-only memory already used for the compressed image. Thus spending more total memory in run-time than doing it completely uncompressed will need...

[gvanem]

Compiling with -DCURL_CA_EMBED and a curl.exe --dump-ca-embed works fine. But
compiling w/o -DCURL_CA_EMBED and the same command prints nothing.

Is this intentional? Why not a "--dump-ca-embed option was disabled at build-time" (as for --manual)

[vszakats]

Thanks for your feedback.

My thinking is that an empty output is a clear sign that there is no embedded CA. Unlike --manual, I expect this output to be more often used in scripts or redirected, instead of being read by humans on the terminal. For those uses an stderr output might be more of a burden than useful (though an errorlevel might help?). All in all, no strong opinion here.

This feature is brand new and I guess time/feedback/contributions will hint what would yet be useful to tweak around it.