Resolve OpenSSL 4.0 build issues - #13482
Closed
cmcfarlen wants to merge 2 commits into
Closed
Conversation
Fedora 45 ships OpenSSL 4.0, which constifies several X509/ASN1 accessors and makes ASN1_STRING opaque. Hardcoding const breaks OpenSSL 1.1.1, where X509_NAME_get_index_by_NID() still takes a non-const X509_NAME, so deduce the pointer and function-pointer types from the accessors instead. That keeps a single source form building on 1.1.1, 3.x and 4.0 with no version macros. Co-authored-by: Jered Floyd <jered@redhat.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates multiple ATS core components, plugins, examples, and tests to build cleanly across OpenSSL 1.1.1, 3.x, and 4.0 by avoiding direct access to newly-opaque ASN1 internals and by letting constness/function-pointer types be deduced from OpenSSL accessors (instead of hardcoding types that differ across versions).
Changes:
- Replace direct
ASN1_STRINGfield access (->data,->length,->type) with accessor APIs (ASN1_STRING_get0_data,ASN1_STRING_length,ASN1_STRING_type). - Update X509/X509_NAME usage to be const-correct across OpenSSL versions via
auto */decltype(...)-based deduction. - Adjust certificate-subject mutation patterns (dup + set back) where OpenSSL now exposes subject names as const.
Reviewed changes
Copilot reviewed 15 out of 15 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| tests/tools/plugins/ssl_client_verify_test.cc | Deduces subject name constness and updates CN extraction to use ASN1 accessors. |
| src/tscore/X509HostnameValidator.cc | Switches hostname validation string checks to ASN1 accessor APIs and deduced subject name constness. |
| src/iocore/net/unit_tests/test_SSLDHParams.cc | Updates self-signed cert generation to dup/replace subject name for OpenSSL 4 constness changes. |
| src/iocore/net/SSLUtils.cc | Makes ASN1 string duplication const-correct and deduces subject name constness for CN extraction. |
| src/iocore/net/SSLNetVConnection.cc | Adjusts debug helper signature to accept const X509_NAME *. |
| src/iocore/net/OCSPStapling.cc | Replaces direct ASN1 buffer access with accessor APIs. |
| src/cripts/Certs.cc | Updates certificate field extraction to use ASN1 accessors and deduced getter types. |
| include/cripts/Certs.hpp | Defines deduced getter typedefs (decltype(&X509_get_...)) to keep function-pointer types portable across OpenSSL versions. |
| src/api/InkAPI.cc | Updates CN extraction types and ASN1 access for OpenSSL 4 compatibility. |
| plugins/lua/ts_lua_client_cert_helpers.h | Updates X509 name helper constness and uses ASN1 accessors for signature formatting. |
| plugins/experimental/txn_box/plugin/src/ts_util.cc | Uses decltype(X509_get_subject_name(nullptr)) to deduce X509_NAME* constness for nid lookup. |
| plugins/experimental/sslheaders/expand.cc | Makes issuer/subject name pointers const and uses ASN1 accessors for signature dump. |
| plugins/experimental/cert_reporting_tool/cert_reporting_tool.cc | Makes subject name pointer const to match OpenSSL accessor changes. |
| plugins/certifier/certifier.cc | Dups subject name before mutation and introduces an RAII type for X509_NAME. |
| example/plugins/c-api/client_context_dump/client_context_dump.cc | Deduces subject name constness for compatibility. |
Drop the unnecessary const cast when duplicating an ASN1 string, assert the subject CN is actually added in the DH params test, build the CN string directly from the ASN1 buffer instead of strndup, and use an explicit deleter for the X509_NAME unique_ptr rather than specializing std::default_delete.
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 15 out of 15 changed files in this pull request and generated no new comments.
Suppressed comments (1)
tests/tools/plugins/ssl_client_verify_test.cc:78
- The SAN (subjectAltName) GEN_DNS path still uses strndup()/free() and then passes the resulting char* to check_name(const std::string&). If strndup() returns nullptr (allocation failure), constructing a std::string from it is UB and can crash. Since you already have pointer+length from OpenSSL, you can avoid the allocation entirely by constructing a std::string directly (same approach as the CN path above).
std::string subj_name{reinterpret_cast<const char *>(ASN1_STRING_get0_data(cn)), static_cast<size_t>(ASN1_STRING_length(cn))};
retval = check_name(subj_name);
}
}
if (!retval) {
3 tasks
maskit
added a commit
to maskit/trafficserver
that referenced
this pull request
Aug 3, 2026
certifier.cc freed the duplicated X509_NAME manually on every path; switch to a scoped_X509_NAME unique_ptr matching the file existing scoped_X509/scoped_EVP_PKEY/scoped_SSL_CTX aliases so no path can forget to free it. ts_util.cc templated ssl_value_for just to defer the parameter type to the caller; decltype(X509_get_subject_name(nullptr)) deduces the same pointer type directly without turning it into a template. Also drop the last C-style cast this series introduced in X509HostnameValidator.cc in favor of reinterpret_cast.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fedora 45 ships OpenSSL 4.0, which breaks the ATS build. This supersedes #13440 by
@jeredfloyd, which had the right diagnosis but did not build on OpenSSL 1.1.1 and
missed several affected files. Jered is credited via
Co-authored-by.Resolves #13427.
Root cause
Diffing the actual headers from OpenSSL 1.1.1f and 4.0.1, exactly one API changes in
a way that can break compilation:
X509_NAME_get_index_by_NIDX509_NAME *const X509_NAME *Others changed only their return type to const in 4.0 (
X509_get_subject_name,X509_get_issuer_name,X509_NAME_get_entry,X509_NAME_ENTRY_get_data,X509_get0_pubkey_bitstr), andASN1_STRINGbecame opaque. Receiving those into aconst-qualified variable is legal on every version.
So hardcoding
constis required on OpenSSL 4 but illegal on 1.1.1 — which is why#13440 failed the Ubuntu 20.04 and FreeBSD 13.1 jobs.
Approach
Let the type be deduced from the accessor rather than hardcoded —
auto *forlocals,
decltype(&X509_get_subject_name)for parameter and function-pointer types.One source form compiles on 1.1.1, 3.x and 4.0 with no version macros and no
const_cast.One case makes this mandatory rather than merely tidy:
X509_getm_notBefore/X509_getm_notAfterchanged const in the opposite direction (they tookconst X509 *in 1.1.1 and takeX509 *in 4.0), so no hardcoded spelling workseverywhere.
Beyond #13440
These are also broken on OpenSSL 4 and were not covered by #13440:
plugins/lua/ts_lua_client_cert_helpers.h,plugins/lua/ts_lua_client_request.ccsrc/cripts/Certs.cc,include/cripts/Certs.hpp— same opaque-ASN1_STRINGissue that Resolve OpenSSL 4.0 build issues #13440 fixed in sslheaders, plus two function-pointer types
src/iocore/net/unit_tests/test_SSLDHParams.ccexample/plugins/c-api/client_context_dump/client_context_dump.cctests/tools/plugins/ssl_client_verify_test.ccTwo fixes here are not purely mechanical:
plugins/certifier/certifier.cc— theX509_NAME_dupapproach needs the copyfreed on the
X509_NAME_add_entry_by_txtfailure path and thedupresultnull-checked. Reworked onto the file's existing
std::unique_ptridiom.test_SSLDHParams.cc— this relied on mutating the subject name in place.Once the accessor returns const you must dup and then call both
X509_set_subject_nameandX509_set_issuer_name, or the generated self-signedcert silently loses its CN.
Verification
Full builds with
BUILD_EXPERIMENTAL_PLUGINS=ONandBUILD_REGRESSION_TESTING=ON,Cripts enabled where the platform's fmt allows it, plus
test_tscoreandtest_net:All on linux/arm64.
Caveats:
test_SSLDHParams.ccis gated behindSSLLIB_IS_AT_LEAST_OPENSSL3, so that changeis only exercised on 3.x/4.0.
minimum — so the Cripts changes are verified on OpenSSL 3.5 and 4.0 only.
src/tscore/unit_tests/test_X509HostnameValidator.ccexists but is not wired intoany CMake target, so
validate_hostnamehas no unit coverage. Pre-existing; leftalone here.