Skip to content

Replace ENGINE support with PROVIDER support on OpenSSL3+ - #13458

Closed
JosiahWI wants to merge 7 commits into
apache:masterfrom
JosiahWI:feat/openssl3-provider-support
Closed

Replace ENGINE support with PROVIDER support on OpenSSL3+#13458
JosiahWI wants to merge 7 commits into
apache:masterfrom
JosiahWI:feat/openssl3-provider-support

Conversation

@JosiahWI

@JosiahWI JosiahWI commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Resolves #13347

JosiahWI added 7 commits July 28, 2026 19:58
Fixes apache#13347

This patch changes the implementation of `SSLPrivateKeyHandler` to use
`SSL_CTX_use_RSAPrivateKey_file` instead of `ENGINE_` APIs, since those
APIs are deprecated in OpenSSL 3.x.
* Rename `load_xxx` to `use_xxx`
* Push `Dbg` message back to `SSLPrivateKeyHandler`
* Move `use_xxx` functions to SSLKeyUtils.{h,cc}
@bneradt

bneradt commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

[approve ci autest 1]

@bneradt bneradt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Remove the remaining OpenSSL 3 ENGINE call

This PR says it resolves #13347, but SSLPostConfigInitialize() still calls ENGINE_load_dynamic(), one of the deprecated calls explicitly listed in that issue. With the current OpenSSL 3 compatibility setting, CMake still detects this API and ATS invokes it whenever proxy.config.ssl.engine.conf_file is configured. Please exclude this call under OPENSSL_IS_OPENSSL3 or replace the initialization path before resolving the issue.

REQUIRE(provider.is_loaded());

TempFile cert{ck.cert_pem};
CHECK(load_key_via_load_certs(cert.get_path(), MockHardwareProvider::URI));

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Preserve provider URIs before filesystem path resolution

This test injects the raw URI directly into CertLoadData, bypassing the production configuration path. load_certs_and_cross_reference_names() passes every ssl_key_name through Layout::relative_to(params->serverKeyPathOnly, keyname), so the default key directory turns pkcs11:... into /.../pkcs11:.... OSSL_STORE_open() consequently receives a filesystem path instead of the provider scheme, making the new provider path unreachable from normal ssl_multicert.yaml configuration. Please preserve recognized store URIs before filesystem resolution and exercise the full normalization path in this test.

}
}
bool result{false};
if (keyPath && keyPath[0] != '\0') {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Keep SSL-secret plugin data authoritative

Trying keyPath before the secret_data returned by SSLSecret::getOrLoadSecret() reverses the documented TS_LIFECYCLE_SSL_SECRET_HOOK contract, where plugin data replaces disk loading. If the configured path contains a stale but parseable key, this helper succeeds and skips the plugin key; the later certificate check can then fail without ever trying the matching secret. This also defeats consistent updates made through TSSslSecretSet(). Please retain secret-data precedence for ordinary file names and use the path-first store lookup only for actual provider URIs.

return false;
}

if (!SSL_CTX_check_private_key(ctx)) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Do not require this comparison for opaque hardware keys

The previous e == nullptr guard was deliberate: hardware-backed ENGINE keys, including HSM/TPM implementations, may support signing without exposing enough key material for SSL_CTX_check_private_key() to compare them. Making the check unconditional rejects those keys in the retained pre-OpenSSL-3 ENGINE path and can likewise reject opaque provider keys. The mock provider does not cover this because it returns an ordinary exportable PEM key. Please preserve the hardware-key distinction or use a validation path that provider/ENGINE-backed opaque keys can support.

#if HAVE_ENGINE_GET_DEFAULT_RSA && HAVE_ENGINE_LOAD_PRIVATE_KEY
// Before providers, a hardware key store was reached through an ENGINE.
// Absent a configured engine there is nothing to ask.
if (ENGINE *e = ENGINE_get_default_RSA(); e != nullptr) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] Release the functional ENGINE reference

ENGINE_get_default_RSA() returns an incremented functional reference that must be released with ENGINE_finish() before it is discarded. Neither the successful early return nor the failure/fallback path releases e, so repeated certificate loads and configuration reloads leak ENGINE references. Please manage this with an RAII deleter or explicitly finish the ENGINE on every path.

@JosiahWI

JosiahWI commented Aug 1, 2026

Copy link
Copy Markdown
Contributor Author

Opting to leave out provider support until there is a need.

@JosiahWI JosiahWI closed this Aug 1, 2026
@JosiahWI
JosiahWI deleted the feat/openssl3-provider-support branch August 1, 2026 17:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remove remaining deprecated OpenSSL 3.0 APIs in SSLUtils.cc (DH and ENGINE)

2 participants