XRootD 5.1.1 CMake Configuration Failure on macOS 11

[krasznaa]

I'm trying to build XRootD 5.1.1 on macOS 11 (Big Sur). As you may know, Apple stopped shipping OpenSSL on its OS with Big Sur. So when I tried to configure the build with ENABLE_CRYPTO=TRUE, I got a fairly understandable error:

CMake Error: The following variables are used in this project, but they are set to NOTFOUND.
Please set them or make sure they are set and tested correctly in the CMake files:
OPENSSL_CRYPTO_LIBRARY
    linked by target "XrdUtils" in directory /Users/krasznaa/ATLAS/projects/externals/build/src/XRootD/src
    linked by target "XrdCl" in directory /Users/krasznaa/ATLAS/projects/externals/build/src/XRootD/src/XrdCl
OPENSSL_SSL_LIBRARY
    linked by target "XrdUtils" in directory /Users/krasznaa/ATLAS/projects/externals/build/src/XRootD/src
    linked by target "XrdCl" in directory /Users/krasznaa/ATLAS/projects/externals/build/src/XRootD/src/XrdCl

CMake Error at src/XrdCrypto.cmake:7 (add_dependencies):
  The dependency target "XrdCryptossl-5" of target "plugins" does not exist.
Call Stack (most recent call first):
  src/CMakeLists.txt:22 (include)

So I thought fine, I'll just try the build with ENABLE_CRYPTO=FALSE. That should fix this. But it doesn't quite. In that setup I still get the "last error" from the previous printout.

CMake Error at src/XrdCrypto.cmake:7 (add_dependencies):
  The dependency target "XrdCryptossl-5" of target "plugins" does not exist.
Call Stack (most recent call first):
  src/CMakeLists.txt:22 (include)

I believe this is an error in the code, here:

https://github.com/xrootd/xrootd/blob/master/src/XrdCrypto.cmake#L7

Since XrdCryptossl-5 is only built with ENABLE_CRYPTO=TRUE, that line should rather be:

if( BUILD_CRYPTO )
  add_dependencies(plugins ${LIB_XRD_CRYPTOSSL})
endif()

[krasznaa]

Of course it would also be nice if src/XrdUtils.cmake and src/XrdCl/CMakeLists.txt would also respect BUILD_CRYPTO with their usage of ${OPENSSL_LIBRARIES}. So that XRootD would build correctly with a missing OpenSSL library, even if ENABLE_CRYPTO=TRUE was used...

[krasznaa]

Hmm... So it's really just impossible to build XRootD without a functional OpenSSL installation at the moment? 😦

/Users/krasznaa/ATLAS/projects/externals/build/src/XRootD/src/XrdTls/XrdTlsContext.cc:20:10: fatal error: 'openssl/bio.h' file not found
#include <openssl/bio.h>
         ^~~~~~~~~~~~~~~
[  7%] Linking CXX static library libXrdFrm.a
[  7%] Built target XrdFrm
[  7%] Building CXX object src/CMakeFiles/XrdUtils.dir/XrdTls/XrdTlsNotary.cc.o
In file included from /Users/krasznaa/ATLAS/projects/externals/build/src/XRootD/src/XrdTls/XrdTlsNotary.cc:31:
/Users/krasznaa/ATLAS/projects/externals/build/src/XRootD/src/./XrdTls/XrdTlsNotary.hh:32:10: fatal error: 'openssl/ssl.h' file not found
#include <openssl/ssl.h>
         ^~~~~~~~~~~~~~~
1 error generated.
make[6]: *** [src/CMakeFiles/XrdUtils.dir/XrdTls/XrdTlsNotary.cc.o] Error 1
make[6]: *** Waiting for unfinished jobs....
/Users/krasznaa/ATLAS/projects/externals/build/src/XRootD/src/XrdTls/XrdTls.cc:32:10: fatal error: 'openssl/err.h' file not found
#include <openssl/err.h>
         ^~~~~~~~~~~~~~~
1 error generated.
make[6]: *** [src/CMakeFiles/XrdUtils.dir/XrdTls/XrdTls.cc.o] Error 1
1 error generated.
make[6]: *** [src/CMakeFiles/XrdUtils.dir/XrdTls/XrdTlsContext.cc.o] Error 1
make[5]: *** [src/CMakeFiles/XrdUtils.dir/all] Error 2
make[4]: *** [all] Error 2

Was this the intention? 😕

[simonmichal]

@krasznaa : thanks for pointing out the inconsistency in the XrdCrypto.cmake file, that said fixing this won't allow you to build xrootd without openssl as also the XrdTls component requires it (this is also not properly reflected in the CMake infrastructure, to be fixed) and it is part of the core not a plug-in (as we are now also supporting secure versions of root/xroot protocol)

You will have to install openssl with brew and then you will need to pass the location of the installed libs and headers using: OPENSSL_INCLUDE_DIR, OPENSSL_SSL_LIBRARY and OPENSSL_CRYPTO_LIBRARY, you can have a look at our CI script for reference:

xrootd/.gitlab-ci.yml

Line 254 in 70066fe

- /usr/local/bin/cmake -D ZLIB_INCLUDE_DIR=/usr/local/Cellar/zlib/1.2.8/include/ -D OPENSSL_INCLUDE_DIR=/usr/local/Cellar/openssl/1.0.2t/include/ -D OPENSSL_SSL_LIBRARY=/usr/local/Cellar/openssl/1.0.2t/lib/libssl.dylib -D OPENSSL_CRYPTO_LIBRARY=/usr/local/Cellar/openssl/1.0.2t/lib/libcrypto.1.0.0.dylib -DCMAKE_INSTALL_PREFIX=../tarball/xrootd ..

[krasznaa]

Okay, I'll just bite the bullet then... Technically I would've been fine with an XRootD installation that would lack SSL cryptography all together. But since that doesn't work anymore (it did work fine with XRootD 4...), I'll be doing this:

https://gitlab.cern.ch/atlas/atlasexternals/-/merge_requests/839/diffs?commit_id=daf925d2275d7eebf989e8911e703521e2c85920

[simonmichal]

@krasznaa : sorry for the late reply but I have been absent for few days; yes this looks about right; I don't think there's anything else we can currently do about it, my apologies for the inconvenience! Can we close this ticket?

[krasznaa]

We can. Though it would be worth thinking of the possibility of making the XRootD code functional without SSL. From what I understood, in the past its "core functionality" was not dependent on that library. Which naively suggests to me that design-wise something is not as great by now as it was in version 4. 🤔

But fine, we can live with the mandatory SSL dependence in ATLAS...