OpenSSL 3.0: Offer TLS providers as an alternative to TLS engines in DNSdist

[fredmorcos]

Short description

OpenSSL 3.0 replaced engines with providers. This change offers loadTLSProvider as an alternative to loadTLSEngine. This is currently used for e.g. HW accelerated crypto: https://github.com/intel/QAT_Engine/blob/master/docs/qat_common.md#openssl-30-provider-support

Providers are disabled by default. The configure flag --enable-tls-providers is available to force-enable them over engines. It is marked as experimental though.

Checklist

I have:

• read the CONTRIBUTING.md document
• compiled this code
• tested this code
• included documentation (including possible behaviour changes)
• documented the code
• added or modified regression test(s)
• added or modified unit test(s)

[fredmorcos]

Regarding build guards (i.e. #ifdefs)

• Add an experimental --enable-tls-providers build flag
• Ensure that loadTLSEngine and related functions are only built when #if defined(HAVE_LIBSSL) && !defined(ENABLE_TLS_PROVIDERS)
• Ensure that loadTLSProvider and related functions are only built when #if defined(HAVE_LIBSSL) && OPENSSL_VERSION_MAJOR >= 3 && defined(ENABLE_TLS_PROVIDERS): This is due to the Intel QAT OpenSSL provider still being experimental.

Documentation

• Documentation needs to be double-checked.
• Which DNSdist version will this be introduced in?

Tests

• Run tests for loadTLSEngine with those changes.
• Add tests for loadTLSProvider.

[rgacogne]

* [ ]  Add an experimental `--enable-tls-provider` build flag

I was pondering doing it the other way around: have a --enable-deprecated-tls-engine that would use the TLS engine interface when building against OpenSSL 3.0.x. It's officially deprecated but it might be the only way to support some functionalities for which a TLS provider is not available.

* [ ]  Which DNSdist [version](https://github.com/PowerDNS/pdns/pull/12423/files#diff-abd4de1691f1c874111fbd923e679855f845526ea13bca77f6084be3d187f7b1R1860) will this be introduced in?

I'm aiming for 1.8.0.

[rgacogne]

I did a quick test by loading the 'legacy' provider and that works. I'll try with QAT later today.

[rgacogne]

I'll try with QAT later today.

So.. Technically I managed to load the QAT provider, and to handle a few TLS connections with it. However it crashed pretty quickly in the QAT provider itself, but I don't think this is on us, the QAT provider is very experimental and from what I read not really tested.
I manually edited the guard to restore the ability to load an OpenSSL engine even with OpenSSL >= 3.0 and it works as expected. So I believe we really need to make it easy to have the ability to use OpenSSL engines even when compiled against OpenSSL >= 3.0, as providers might not be production-ready for a while. How about adding --enable-tls-providers and --enable-tls-engines? I'm not even sure we need to make them exclusive, to be honest, so we might be able to just implement --enable-tls-engines, disabled by default, and only enable providers when OpenSSL >= 3.0?

[fredmorcos]

so we might be able to just implement --enable-tls-engines, disabled by default, and only enable providers when OpenSSL >= 3.0?

I'm not sure this is a good idea given the sorry state of the Intel QAT provider (which is the only use-case I know of so far). I would prefer having engines enabled by default and --enable-tls-providers which only works with OpenSSL >= 3.0. Is that OK?

[rgacogne]

so we might be able to just implement --enable-tls-engines, disabled by default, and only enable providers when OpenSSL >= 3.0?

I'm not sure this is a good idea given the sorry state of the Intel QAT provider (which is the only use-case I know of so far). I would prefer having engines enabled by default and --enable-tls-providers which only works with OpenSSL >= 3.0. Is that OK?

Yes, I think you are right that this is a better default. I'm a bit sad because it means we will have some remaining warnings by default when building with OpenSSL >= 3.0, because of the deprecated engines, but in practice this is likely what most users are going to want for now.