Build failed on RHEL9 with OpenSSL 3

[majestix508]

I was building the main branch with:
cmake -DBUILD_ONLY="sns" -DBUILD_SHARED_LIBS=OFF ../
But at the make-run I get Segmentation Faults on sns-gen-tests so I tried to build with:

cmake -DBUILD_ONLY="sns" ../
then on "make -j 4 VERBOSE=1" I get this output:

cd /home/someuser/aws-sdk-cpp/build/generated/tests/sns-gen-tests && /usr/bin/cmake -E cmake_link_script CMakeFiles/sns-gen-tests.dir/link.txt --verbose=1

/usr/bin/c++ -O3 -DNDEBUG @CMakeFiles/sns-gen-tests.dir/objects1.rsp -o sns-gen-tests  -Wl,-rpath,/home/someuser/aws-sdk-cpp/build/tests/testing-resources:/home/someuser/aws-sdk-cpp/build/generated/src/aws-cpp-sdk-sns:/home/someuser/aws-sdk-cpp/build/src/aws-cpp-sdk-core ../../../tests/testing-resources/libtesting-resources.so ../../src/aws-cpp-sdk-sns/libaws-cpp-sdk-sns.so ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so ../../../crt/aws-crt-cpp/libaws-crt-cpp.a ../../../crt/aws-crt-cpp/crt/aws-c-mqtt/libaws-c-mqtt.a ../../../lib/libaws-c-event-stream.a ../../../crt/aws-crt-cpp/crt/aws-c-s3/libaws-c-s3.a ../../../crt/aws-crt-cpp/crt/aws-c-auth/libaws-c-auth.a ../../../crt/aws-crt-cpp/crt/aws-c-http/libaws-c-http.a ../../../crt/aws-crt-cpp/crt/aws-c-io/libaws-c-io.a ../../../lib/libs2n.a /usr/lib64/libcrypto.so ../../../crt/aws-crt-cpp/crt/aws-c-compression/libaws-c-compression.a ../../../crt/aws-crt-cpp/crt/aws-c-cal/libaws-c-cal.a /usr/lib64/libcrypto.so ../../../crt/aws-crt-cpp/crt/aws-c-sdkutils/libaws-c-sdkutils.a ../../../crt/aws-crt-cpp/crt/aws-checksums/libaws-checksums.a ../../../crt/aws-crt-cpp/crt/aws-c-common/libaws-c-common.a -lpthread -ldl -lm -lrt 
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `OPENSSL_free'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_CIPHER_CTX_key_length'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `sk_pop_free_ex'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `SSLeay'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `X509_NAME_hash'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_CIPHER_CTX_cleanup'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_aead_aes_256_gcm_tls13'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `AWSLC_thread_local_clear'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_AEAD_CTX_seal'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `sk_new_null'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `BN_num_bytes'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_AEAD_CTX_open'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_aead_aes_128_gcm'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_AEAD_CTX_cleanup'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_MD_CTX_destroy'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_aead_chacha20_poly1305'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_PKEY_get1_tls_encodedpoint'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `OPENSSL_malloc'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `AWSLC_thread_local_shutdown'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `sk_num'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_AEAD_CTX_free'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `sk_value'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_MD_CTX_create'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `sk_free'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `FIPS_mode'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_PKEY_base_id'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_aead_aes_256_gcm'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_AEAD_CTX_init'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_CIPHER_CTX_init'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `sk_push'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_AEAD_CTX_zero'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_aead_aes_256_gcm_tls12'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_aead_aes_128_gcm_tls13'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `SSLeay_version'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_MD_size'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_aead_aes_128_gcm_tls12'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_PKEY_size'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `CRYPTO_tls1_prf'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_PKEY_set1_tls_encodedpoint'
/usr/bin/ld: ../../../src/aws-cpp-sdk-core/libaws-cpp-sdk-core.so: undefined reference to `EVP_MD_CTX_size'
collect2: error: ld returned 1 exit status
make[2]: *** [generated/tests/sns-gen-tests/CMakeFiles/sns-gen-tests.dir/build.make:134: generated/tests/sns-gen-tests/sns-gen-tests] Error 1
make[2]: Leaving directory '/home/someuser/aws-sdk-cpp/build'
make[1]: *** [CMakeFiles/Makefile2:939: generated/tests/sns-gen-tests/CMakeFiles/sns-gen-tests.dir/all] Error 2
make[1]: Leaving directory '/home/someuser/aws-sdk-cpp/build'
make: *** [Makefile:156: all] Error 2

Expected Behavior

build should finish and tests should pass

Current Behavior

failed on build - see description

Reproduction Steps

RHEL 9.2 (Plow)
gcc 11.3.1
openssl 1.3.0

git clone --recurse-submodules https://github.com/aws/aws-sdk-cpp
cd aws-sdk-cpp
mkdir build;cd build

cmake -DBUILD_ONLY="sns" -DBUILD_SHARED_LIBS=OFF ../
or
cmake -DBUILD_ONLY="sns" ../

make -j 4

Possible Solution

No response

Additional Information/Context

No response

AWS CPP SDK version used

1.11.137

Compiler and Version used

gcc 11.3.1

Operating System and version

RHEL 9.2

[SergeyRyabinin]

Hi @majestix508 ,

Thank you for posting this issue.
We are using a custom FindCrypto cmake module which might give some issues. It seems that crypto lib headers found do not match crypto lib binaries found.

Could you please also post the output of the cmake configure stage i.e. cmake -DBUILD_ONLY="sns" ../?

Best regards,
Sergey

[majestix508]

`cmake -DBUILD_ONLY="sns" ../
CMake Warning at CMakeLists.txt:9 (message):
In 1.11 releases, we are releasing experimental alternative building
mode.By setting -DLEGACY_MODE=OFF you can test our advances in modern CMake
building and provide early feedback. The legacy support is set by default
in 1.11, when you complete build updating scripts please update the build
flags as mentioned in README.md and set -DLEGACY_BUILD=OFF. The legacy
support will be removed at 1.12.0 release.

-- TARGET_ARCH not specified; inferring host OS to be platform compilation target
-- Building AWS libraries as shared objects
-- Generating linux build config
-- Building project version: 1.11.137
-- The CXX compiler identification is GNU 11.3.1
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found PythonInterp: /usr/bin/python3.9 (found version "3.9.16")
-- Found ZLIB: /usr/lib64/libz.so (found version "1.2.11")
-- Zlib library: /usr/lib64/libz.so
-- Encryption: LibCrypto
-- Found crypto: /usr/lib64/libcrypto.so
-- LibCrypto Include Dir: /usr/local/include
-- LibCrypto Shared Lib: /usr/lib64/libcrypto.so
-- LibCrypto Static Lib: /usr/local/lib64/libcrypto.a
-- Looking for C++ include pthread.h
-- Looking for C++ include pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Http client: Curl
-- Found CURL: /usr/lib64/libcurl.so (found version "7.76.1")
-- Curl include directory: /usr/include
-- Curl library: /usr/lib64/libcurl.so
-- Performing Test HAVE_ATOMICS_WITHOUT_LIBATOMIC
-- Performing Test HAVE_ATOMICS_WITHOUT_LIBATOMIC - Success
-- AWS CRT C++ 0.21.0
-- The C compiler identification is GNU 11.3.1
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
CMake Warning (dev) at crt/aws-crt-cpp/crt/aws-c-common/CMakeLists.txt:5 (option):
Policy CMP0077 is not set: option() honors normal variables. Run "cmake
--help-policy CMP0077" for policy details. Use the cmake_policy command to
set the policy and suppress this warning.

For compatibility with older versions of CMake, option is clearing the
normal variable 'STATIC_CRT'.
This warning is for project developers. Use -Wno-dev to suppress it.

-- CMake 3.20.2
-- Performing Test AWS_HAVE_GCC_OVERFLOW_MATH_EXTENSIONS
-- Performing Test AWS_HAVE_GCC_OVERFLOW_MATH_EXTENSIONS - Success
-- Performing Test AWS_HAVE_WINAPI_DESKTOP
-- Performing Test AWS_HAVE_WINAPI_DESKTOP - Failed
-- Performing Test AWS_ARCH_INTEL
-- Performing Test AWS_ARCH_INTEL - Success
-- Performing Test AWS_ARCH_ARM64
-- Performing Test AWS_ARCH_ARM64 - Failed
-- Performing Test AWS_ARCH_ARM32
-- Performing Test AWS_ARCH_ARM32 - Failed
-- Performing Test AWS_HAVE_GCC_INLINE_ASM
-- Performing Test AWS_HAVE_GCC_INLINE_ASM - Success
-- Performing Test AWS_HAVE_AUXV
-- Performing Test AWS_HAVE_AUXV - Success
-- Performing Test AWS_HAVE_EXECINFO
-- Performing Test AWS_HAVE_EXECINFO - Success
-- Performing Test AWS_HAVE_LINUX_IF_LINK_H
-- Performing Test AWS_HAVE_LINUX_IF_LINK_H - Success
-- Performing Test HAVE_M_AVX2_FLAG
-- Performing Test HAVE_M_AVX2_FLAG - Success
-- Performing Test HAVE_AVX2_INTRINSICS
-- Performing Test HAVE_AVX2_INTRINSICS - Success
-- Performing Test HAVE_MM256_EXTRACT_EPI64
-- Performing Test HAVE_MM256_EXTRACT_EPI64 - Success
-- Looking for pthread_mutexattr_init
-- Looking for pthread_mutexattr_init - not found
-- Performing Test HAS_FPIC_FLAG
-- Performing Test HAS_FPIC_FLAG - Success
-- Performing Test HAS_64BIT_FILE_OFFSET_BY_DEFAULT
-- Performing Test HAS_64BIT_FILE_OFFSET_BY_DEFAULT - Success
-- Looking for fseeko
-- Looking for fseeko - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stdbool.h
-- Looking for stdbool.h - found
-- Performing Test HAS_WGNU
-- Performing Test HAS_WGNU - Failed
-- Performing Test HAVE_SYSCONF
-- Performing Test HAVE_SYSCONF - Success
-- Looking for pthread_attr_setaffinity_np
-- Looking for pthread_attr_setaffinity_np - found
-- Performing Test PTHREAD_SETNAME_TAKES_2ARGS
-- Performing Test PTHREAD_SETNAME_TAKES_2ARGS - Success
-- Performing Test PTHREAD_GETNAME_TAKES_2ARGS
-- Performing Test PTHREAD_GETNAME_TAKES_2ARGS - Failed
-- Performing Test PTHREAD_GET_NAME_TAKES_2ARGS
-- Performing Test PTHREAD_GET_NAME_TAKES_2ARGS - Failed
-- Performing Test PTHREAD_GETNAME_TAKES_3ARGS
-- Performing Test PTHREAD_GETNAME_TAKES_3ARGS - Success
-- Building SIMD base64 decoder
-- Detected CMAKE_SYSTEM_PROCESSOR as x86_64
-- Detected 64-Bit system
-- Using libcrypto from the cmake path
-- CMAKE_AR found: /usr/bin/ar
-- CMAKE_RANLIB found: /usr/bin/ranlib
-- CMAKE_OBJCOPY found: /usr/bin/objcopy
-- feature S2N_ATOMIC_SUPPORTED: TRUE
-- feature S2N_CLONE_SUPPORTED: TRUE
-- feature S2N_CPUID_AVAILABLE: TRUE
-- feature S2N_DIAGNOSTICS_POP_SUPPORTED: TRUE
-- feature S2N_DIAGNOSTICS_PUSH_SUPPORTED: TRUE
-- feature S2N_EXECINFO_AVAILABLE: TRUE
-- feature S2N_FALL_THROUGH_SUPPORTED: TRUE
-- feature S2N_FEATURES_AVAILABLE: TRUE
-- feature S2N_KYBER512R3_AVX2_BMI2_SUPPORTED: TRUE
-- feature S2N_KYBER512R3_M256_INTRINSICS_SUPPORTED: TRUE
-- feature S2N_LIBCRYPTO_SUPPORTS_EVP_MD5_SHA1_HASH: TRUE
-- feature S2N_LIBCRYPTO_SUPPORTS_EVP_MD_CTX_SET_PKEY_CTX: TRUE
-- feature S2N_LIBCRYPTO_SUPPORTS_EVP_RC4: TRUE
-- feature S2N_LIBCRYPTO_SUPPORTS_HKDF: FALSE
-- feature S2N_LIBCRYPTO_SUPPORTS_KYBER: FALSE
-- feature S2N_MADVISE_SUPPORTED: TRUE
-- feature S2N_MINHERIT_SUPPORTED: FALSE
-- feature S2N___RESTRICT__SUPPORTED: TRUE
-- feature S2N_STACKTRACE: TRUE
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- feature S2N_KYBER512R3_AVX2_BMI2: TRUE
-- Found OpenSSL: /usr/lib64/libcrypto.so (found version "3.0.7")
-- Using libcrypto from system: /usr/lib64/libcrypto.so
-- Add s3-crt:s3 to C2J_SPECIAL_NAME_LIST
-- Considering sns
-- Looking for pathconf
-- Looking for pathconf - found
-- Looking for umask
-- Looking for umask - found
-- Adding sns to SDK build
-- Adding core to SDK build
-- Updating version info to 1.11.137
-- Custom memory management disabled
-- Performing Test CURL_HAS_H2
-- Performing Test CURL_HAS_H2 - Success
-- Performing Test CURL_HAS_TLS_PROXY
-- Performing Test CURL_HAS_TLS_PROXY - Success
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- SKIPPING:Threads::Threads
-- Configuring done
-- Generating done
-- Build files have been written to: /home/apatecstockinger/aws-sdk-cpp/build
`

[majestix508]

Found by cmake -> openssl 3.0.7

-- Encryption: LibCrypto
-- Found crypto: /usr/lib64/libcrypto.so
-- LibCrypto Include Dir: /usr/local/include
-- LibCrypto Shared Lib: /usr/lib64/libcrypto.so
-- LibCrypto Static Lib: /usr/local/lib64/libcrypto.a

on RHEL9 these Packages from the standard repo are installed:

sudo dnf list installed|grep ssl
compat-openssl11.x86_64                       1:1.1.1k-4.el9_0                 @rhel-9-for-x86_64-appstream-rpms  
openssl.x86_64                                1:3.0.7-17.el9_2                 @rhel-9-for-x86_64-baseos-rpms     
openssl-devel.x86_64                          1:3.0.7-17.el9_2                 @rhel-9-for-x86_64-appstream-rpms  
openssl-libs.x86_64                           1:3.0.7-17.el9_2                 @rhel-9-for-x86_64-baseos-rpms     
openssl-pkcs11.x86_64                         0.4.11-7.el9                     @anaconda                          

[majestix508]

ok I think I found out what happened:
I tried a couple of cmake settings - one was USE_OPENSSL=FALSE
this doesnt work for me so I tried other settings, but I never did a make uninstall so the libs stayed in /usr/local/lib64
My output from cmake above shows this:

-- Encryption: LibCrypto
-- Found crypto: /usr/lib64/libcrypto.so
-- LibCrypto Include Dir: /usr/local/include
-- LibCrypto Shared Lib: /usr/lib64/libcrypto.so
-- LibCrypto Static Lib: /usr/local/lib64/libcrypto.a

cmake mixes them up using the librypto from /usr/lib64 and includes from /usr/local/include and the static lib from /usr/local/lib64

I removed everything with make uninstall and cleaned up the /usr/local directories
the cmake -DBUILD_ONLY="sns" shows:

-- Found crypto: /usr/lib64/libcrypto.so  
-- LibCrypto Include Dir: /usr/include
-- LibCrypto Shared Lib:  /usr/lib64/libcrypto.so
-- LibCrypto Static Lib:  crypto_STATIC_LIBRARY-NOTFOUND

so I can compile now with shared libraries, but not static. I think this is caused by RHEL9 cause they don't provide a libcrypto-static package anymore...

[jmklix]

@majestix508 did you have any other problems building this sdk?

[majestix508]

Is it possible to compile with static lib on RHEL9?