oscap xccdf eval segfaults with xmlsec1 >= 1.3

[p5]

oscap xccdf eval segfaults (SIGSEGV, exit 139) when linked against xmlsec1 >= 1.3. The crash occurs during XCCDF rule evaluation, even on unsigned datastreams.

Trying to gather improved error logs.

AI Root Cause Analysis

Details

Summary

_oscap_signature_validate_doc() in src/source/signature.c calls xmlSecCryptoShutdown(), xmlSecCryptoAppShutdown(), and xmlSecShutdown() after every signature validation — including on unsigned documents where validation is a no-op ("Signature node not found").

In xmlsec1 1.3, xmlSecCryptoAppShutdown() calls OPENSSL_cleanup(), which irreversibly destroys all process-global OpenSSL state including threading locks. Any subsequent OpenSSL usage in the same process segfaults.

In xmlsec1 1.2, the shutdown path did not call OPENSSL_cleanup(), so this was harmless.

Crash Path

Signature validation (main thread):
  _oscap_signature_validate_doc()
    → xmlSecInit() / xmlSecCryptoAppInit() / xmlSecCryptoInit()
    → "Signature node not found" (unsigned datastream)
    → xmlSecCryptoShutdown()
    → xmlSecCryptoAppShutdown()       ← calls OPENSSL_cleanup()
    → xmlSecShutdown()

Probe evaluation (worker thread, later):
  rpminfo_probe_main()
    → rpmtsInitIterator()
      → ... any OpenSSL call ...
        → CRYPTO_THREAD_write_lock()
          → pthread_rwlock_wrlock()   ← SIGSEGV (lock destroyed)

Backtrace

#0  pthread_rwlock_wrlock@GLIBC_2.2.5 () from libc.so.6
#1  CRYPTO_THREAD_write_lock () from libcrypto.so.3
#2  ERR_get_next_error_library () from libcrypto.so.3
#3  provider_activate_fallbacks () from libcrypto.so.3
#4  OSSL_PROVIDER_available () from libcrypto.so.3
...
#12 rpmDigestBundleAddID () from librpmio.so.10
#13 rpmvsInitRange () from librpm.so.10
#14 headerCheck () from librpm.so.10
#15 rpmdbNextIterator () from librpm.so.10
...
#19 rpminfo_probe_main () from libopenscap.so.33
#20 probe_worker () from libopenscap.so.33
#21 probe_worker_runfn () from libopenscap.so.33

Evidence

• oscap xccdf eval → exit 139 (segfault)
• oscap xccdf eval --skip-signature-validation → works (exit 0 or 2)
• oscap oval eval → works (no signature validation path)
• Downgrading xmlsec1 from 1.3.11 to 1.2.42 → works

Suggested Fix

• Move xmlSecInit() / xmlSecCryptoAppInit() / xmlSecCryptoInit() to a one-time-per-process initialization (e.g. pthread_once)
• Remove xmlSecCryptoShutdown(), xmlSecCryptoAppShutdown(), and xmlSecShutdown() from the cleanup block — these destroy process-global state that the rest of the process relies on
• Per-validation resources (xmlSecDSigCtx, xmlSecKeysMngr) are already cleaned up correctly and should remain as-is

Environment

• openscap 1.4.4
• xmlsec1 1.3.11 (xmlsec1 1.2.42 is not affected)
• OpenSSL 3.5.6
• rpm 6.0.1, rpm-sequoia 1.10.1 (crash site, but any OpenSSL consumer would crash)