Ubuntu 20.04, OpenSSL 1.1.1g: DH_compute_key_padded extern vs static

[Axel-Naumann]

Hi,

Building xrootd 5.3.1 (as ROOT builtin) with openssl 1.1.1g (also ROOT builtin) causes:

/home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/XrdCrypto/XrdCryptosslCipher.cc:140:12: error: ‘int DH_compute_key_padded(unsigned char*, const BIGNUM*, DH*)’ was declared ‘extern’ and later ‘static’ [-fpermissive]
  140 | static int DH_compute_key_padded(unsigned char *key, const BIGNUM *pub_key, DH *dh)
      |            ^~~~~~~~~~~~~~~~~~~~~
In file included from /home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/./XrdCrypto/XrdCryptosslCipher.hh:40,
                 from /home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/XrdCrypto/XrdCryptosslCipher.cc:38:
/home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/dh.h:153:5: note: previous declaration of ‘int DH_compute_key_padded(unsigned char*, const BIGNUM*, DH*)’
  153 | int DH_compute_key_padded(unsigned char *key, const BIGNUM *pub_key, DH *dh);
      |     ^~~~~~~~~~~~~~~~~~~~~
/home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/XrdCrypto/XrdCryptosslCipher.cc: In function ‘int DH_compute_key_padded(unsigned char*, const BIGNUM*, DH*)’
:
/home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/XrdCrypto/XrdCryptosslCipher.cc:143:12: error: invalid use of incomplete type ‘DH’ {aka ‘struct dh_st’}
  143 |     rv = dh->meth->compute_key(key, pub_key, dh);
      |            ^~
In file included from /home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/evp.h:14,
                 from /home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/./XrdCrypto/XrdCryptosslCipher.hh:39,
                 from /home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/XrdCrypto/XrdCryptosslCipher.cc:38:
/home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/ossl_typ.h:104:16: note: forward declaration of ‘DH’ {aka ‘struct dh_st’}
  104 | typedef struct dh_st DH;
      |                ^~~~~
In file included from /home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/asn1.h:23,
                 from /home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/objects.h:15,
                 from /home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/evp.h:28,
                 from /home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/./XrdCrypto/XrdCryptosslCipher.hh:39,
                 from /home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/XrdCrypto/XrdCryptosslCipher.cc:38:
/home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/XrdCrypto/XrdCryptosslCipher.cc:146:26: error: invalid use of incomplete type ‘DH’ {aka ‘struct dh_st’}
  146 |     pad = BN_num_bytes(dh->p) - rv;
      |                          ^~
/home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/bn.h:181:40: note: in definition of macro ‘BN_num_bytes’
  181 | # define BN_num_bytes(a) ((BN_num_bits(a)+7)/8)
      |                                        ^
In file included from /home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/evp.h:14,
                 from /home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/./XrdCrypto/XrdCryptosslCipher.hh:39,
                 from /home/ubuntu/build/root/master/rwd/builtins/xrootd/XROOTD-prefix/src/XROOTD/src/XrdCrypto/XrdCryptosslCipher.cc:38:
/home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/ossl_typ.h:104:16: note: forward declaration of ‘DH’ {aka ‘struct dh_st’}
  104 | typedef struct dh_st DH;
      |                ^~~~~

Might be related to #1558

[simonmichal]

@Axel-Naumann : are you building opelssl from sources or are you using the package available on the platform?
Could you check if the include/openssl/dh.h header is present and if it contains the DH_compute_key_padded signature?

[Axel-Naumann]

In this case we are building openssl during the build of ROOT. As the error message shows, the signature in that file is present:

/home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/openssl/dh.h:153:5: note: previous declaration of ‘int DH_compute_key_padded(unsigned char*, const BIGNUM*, DH*)’
  153 | int DH_compute_key_padded(unsigned char *key, const BIGNUM *pub_key, DH *dh);

[simonmichal]

Is the /home/ubuntu/build/root/master/rwd/builtins/openssl/OPENSSL-prefix/include/ added in the cmake infrastructure to those the compiler uses to search for include files? In principle the signature should have been detected by:

xrootd/cmake/FindOpenSSL.cmake

Lines 85 to 89 in 1abd369

	check_function_exists(DH_compute_key_padded HAVE_DH_PADDED_FUNC)
	check_symbol_exists(
	DH_compute_key_padded
	${OPENSSL_INCLUDE_DIR}/openssl/dh.h
	HAVE_DH_PADDED_SYMB)

Does similar builds (building xrootd from sources + building openssl from sources) work on other platforms?

[gganis]

Hi, I could reproduce it in standalone. The problem is that, when forcing static openssl libraries (as ROOT does) the 'check_function_exists' and 'check_symbol_exists' do not detect what we expect (at least as implemented in xrootd/cmake/FindOpenSSL.cmake):

758:HAVE_DH_PADDED_FUNC:INTERNAL=
760:HAVE_DH_PADDED_SYMB:INTERNAL=

(content of CMakeCache.txt)

[Axel-Naumann]

Any news on this, @simonmichal ?

[abh3]

@Axel-Naumann A pr is about to be merged so we are almost at the finish line.

[simonmichal]

@Axel-Naumann : I just merged #1596 (@gganis : again, thanks a lot :-)

[Axel-Naumann]

Excellent, thanks @gganis and everyone else involved! What's the timeline for the next xrootd client release (for us to figure out how we can roll this fix out)?

[simonmichal]

@Axel-Naumann : we plan to start the release process for 5.4.1 (RC1) next week, it usually takes 2-3 weeks until we have the final release.

[abh3]

I believe this problem has moved into deployment status as the code itself has been fixed. If I am wrong, please reopen the ticket.