sss Authentication on Centos9 Stream does not work with Xrootd 5.4.3-rc3

[apeters1971]

A simple xrdfs stat using sss authentication reveals the problem:

(gdb) run eoshome-a.cern.ch stat /eos/
Starting program: /opt/eos/xrootd/bin/xrdfs eoshome-a.cern.ch stat /eos/
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
[New Thread 0x7ffff6ff1640 (LWP 2148217)]
[New Thread 0x7ffff67f0640 (LWP 2148218)]
[New Thread 0x7ffff5fef640 (LWP 2148219)]
[New Thread 0x7ffff57ee640 (LWP 2148220)]
[New Thread 0x7ffff4fed640 (LWP 2148221)]
sec_Client: protocol request for host eoshome-a.cern.ch token='&P=krb5,xrootd/eoshome.cern.ch@CERN.CH&P=gsi,v:10400,c:ssl,ca:5168735f.0|4339b4bc.0&P=sss,0.13:/etc/eos.keytab&P=unixgin%'
sec_PM: Loaded krb5 protocol object from libXrdSeckrb5.so
sec_PM: Using krb5 protocol, args='xrootd/eoshome.cern.ch@CERN.CH'
Seckrb5: getCredentials
Seckrb5: context lock
Seckrb5: context locked
Seckrb5: credentials cache unset
Seckrb5: init context
Seckrb5: cc set default name
Seckrb5: cc default
Seckrb5: get_krbCreds: err copying client name to creds; No credentials cache found
sec_Client: protocol request for host eoshome-a.cern.ch token='&P=gsi,v:10400,c:ssl,ca:5168735f.0|4339b4bc.0&P=sss,0.13:/etc/eos.keytab&P=unixgin%'
sec_PM: Loaded gsi protocol object from libXrdSecgsi.so
Secgsi -------------------------------------------------------------------
Secgsi Mode: client
Secgsi Debug: 1
Secgsi CA dir: /etc/grid-security/certificates/
Secgsi CA verification level: verifyss
Secgsi CRL dir: /etc/grid-security/certificates/
Secgsi CRL extension: .r0
Secgsi CRL check level: try
Secgsi CRL refresh time: 86400
Secgsi Certificate: /root/.globus/usercert.pem
Secgsi Key: /root/.globus/userkey.pem
Secgsi Proxy file: /tmp/x509up_u0
Secgsi Proxy validity: 12:00
Secgsi Proxy dep length: 0
Secgsi Proxy bits: 512
Secgsi Proxy sign option: 1
Secgsi Proxy delegation option: 0
Secgsi Pure Cert/Key authentication allowed
Secgsi Allowed server names: [*/]<target host name>[/*]
Secgsi Crypto modules: ssl
Secgsi Ciphers: aes-128-cbc:bf-cbc:des-ede3-cbc
Secgsi MDigests: sha1:md5
Secgsi Trusting DNS for hostname checking
Secgsi -------------------------------------------------------------------
sec_PM: Using gsi protocol, args='v:10400,c:ssl,ca:5168735f.0|4339b4bc.0'
220614 13:14:36 2148217 cryptossl_X509::CertType: certificate has 10 extensions
220614 13:14:36 2148217 secgsi_VerifyCA: Warning: CA certificate not self-signed and integrity not checked: assuming OK (5168735f.0)
220614 13:14:36 2148217 cryptossl_X509::CertType: certificate has 10 extensions
220614 13:14:36 2148217 secgsi_QueryProxy: problems initializing proxy via external shell
sec_Client: protocol request for host eoshome-a.cern.ch token='&P=sss,0.13:/etc/eos.keytab&P=unixgin%'
sec_PM: Loaded sss protocol object from libXrdSecsss.so
[New Thread 0x7fffeffff640 (LWP 2148222)]
sec_sss: Client keytab='/etc/eos/fuse.sss.keytab'
sec_PM: Using sss protocol, args='0.13:/etc/eos.keytab'
sec_sss: getCreds: 0 ud: '' ip: '[::ffff:188.185.9.6]:33840'
sec_sss: Encode keyid: 6752069312392986625 bytes 187

Thread 2 "xrdfs" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff6ff1640 (LWP 2148217)]
0x00007ffff736cf78 in EVP_CIPHER_CTX_set_key_length (c=0x7ffff00d2bb0, keylen=32) at crypto/evp/evp_enc.c:979
Downloading source file /usr/src/debug/openssl-3.0.1-18.el9.x86_64/crypto/evp/evp_enc.c...
979        if (c->cipher->prov != NULL) {

Full stack:

(gdb) where
#0  0x00007ffff736cf78 in EVP_CIPHER_CTX_set_key_length (c=0x7ffff00d2bb0, keylen=32) at crypto/evp/evp_enc.c:979
#1  0x00007ffff47d155e in XrdCryptoLite_bf32::Encrypt (this=<optimized out>, key=0x7ffff6fef578 "\352=5\322\335d\255G\224\242\342\367I\237\323\304^\016\212Y\v\fO\370\242\004\234\276\302Dcl", keyLen=32, src=<optimized out>, srcLen=183, dst=0x7ffff0024ab0 "", 
    dstLen=187) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCrypto/XrdCryptoLite_bf32.cc:157
#2  0x00007ffff47db10d in XrdSecProtocolsss::Encode (this=this@entry=0x7ffff0129930, einfo=einfo@entry=0x7ffff6fef850, encKey=..., rrHdr=rrHdr@entry=0x7ffff6fef480, rrDHdr=0x7ffff0128c30, dLen=dLen@entry=183)
    at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdSecsss/XrdSecProtocolsss.cc:501
#3  0x00007ffff47dc5e5 in XrdSecProtocolsss::getCredentials (this=0x7ffff0129930, parms=<optimized out>, einfo=0x7ffff6fef850) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdSecsss/XrdSecProtocolsss.cc:693
#4  0x00007ffff7eaf4d5 in XrdCl::XRootDTransport::GetCredentials (this=<optimized out>, credentials=@0x7ffff6ff0108: 0x0, hsData=0x7ffff0000b60, info=0x48b020) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCl/XrdClXRootDTransport.cc:2592
#5  0x00007ffff7eafd5a in XrdCl::XRootDTransport::DoAuthentication (this=0x48a2b0, hsData=0x7ffff0000b60, info=0x48b020) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCl/XrdClXRootDTransport.cc:2345
#6  0x00007ffff7eb2723 in XrdCl::XRootDTransport::HandShakeMain (this=0x48a2b0, handShakeData=0x7ffff0000b60, channelData=...) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCl/XrdClXRootDTransport.cc:539
#7  0x00007ffff7eb2a71 in XrdCl::XRootDTransport::HandShake (this=0x48a2b0, handShakeData=0x7ffff0000b60, channelData=...) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCl/XrdClXRootDTransport.cc:439
#8  0x00007ffff7f2e5c8 in XrdCl::AsyncSocketHandler::HandleHandShake (this=0x48b560, msg=...) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCl/XrdClAsyncSocketHandler.cc:539
#9  0x00007ffff7f2e99b in XrdCl::AsyncSocketHandler::OnReadWhileHandshaking (this=0x48b560) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCl/XrdClAsyncSocketHandler.cc:527
#10 0x00007ffff7f2eda5 in XrdCl::AsyncSocketHandler::Event (this=0x48b560, type=1 '\001') at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCl/XrdClAsyncSocketHandler.cc:227
#11 0x00007ffff7e9c686 in (anonymous namespace)::SocketCallBack::Event (this=0x48bbd0, chP=<optimized out>, cbArg=<optimized out>, evFlags=<optimized out>) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdCl/XrdClPollerBuiltIn.cc:83
#12 0x00007ffff7c634f7 in XrdSys::IOEvents::Poller::CbkXeq (this=0x489010, cP=0x48bbf0, events=1, eNum=<optimized out>, eTxt=<optimized out>) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdSys/XrdSysIOEvents.cc:721
#13 0x00007ffff7c647ec in XrdSys::IOEvents::PollE::Dispatch (this=this@entry=0x489010, cP=0x48bbf0, pollEv=<optimized out>) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/./XrdSys/XrdSysIOEventsPollE.icc:275
#14 0x00007ffff7c649e8 in XrdSys::IOEvents::PollE::Begin (this=0x489010, syncsem=<optimized out>, retcode=<optimized out>, eTxt=<optimized out>) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/./XrdSys/XrdSysIOEventsPollE.icc:230
#15 0x00007ffff7c613cd in XrdSys::IOEvents::BootStrap::Start (parg=0x7fffffffcf60) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdSys/XrdSysIOEvents.cc:149
#16 0x00007ffff7c699a8 in XrdSysThread_Xeq (myargs=0x485500) at /usr/src/debug/eos-xrootd-5.4.4-1.el9.x86_64/src/XrdSys/XrdSysPthread.cc:86
#17 0x00007ffff778e83a in start_thread (arg=<optimized out>) at pthread_create.c:443
#18 0x00007ffff772e4c0 in clone3 () at ../sysdeps/unix/sysv/linux/x86_64/clone3.S:81
(gdb) 

[smithdh]

The problem is at by default openssl 3 does not enable the "legacy provider" that would allow certain algorithms to be used. This includes the blowfish cipher used by sss. There is the possibility of enabling it system wide: I've tested that by applying the setting on centos9s, it allows a machine to act as a client and authenticate with an centos7 server. The setting should also allow a server to run on centos9s and use sss. The change is:

In /etc/ssl/openssl.cnf ensure these lines are uncommented:

[provider_sect]
default = default_sect
legacy = legacy_sect

[default_sect]
activate = 1

[legacy_sect]
activate = 1

The above lines exist in the default conf file, but some are commented out.

The configuration file approach may not be suitable for all situations, particularly when users are the ones running the client on machines where we can't reasonably expect the system wide configuration to be changed. I'll prepare a PR to programatically enable the legacy provider in XrdCryptoLite_bf32.

[smithdh]

Thank you for merging; I'll close the ticket now.