Revamp how libomp.so is found #853
Conversation
An upstream llvm change enables LLVM_ENABLE_PER_TARGET_RUNTIME_DIR by default for the openmp build. This installs the openmp libraries into /opt/rocm-ver/llvm/lib/x86_64-unknown-linux-gnu instead of /opt/rocm-ver/llvm/lib. Currenty, hipBLAS only uses /lib. Since hipBLAS uses gcc by default, find_package(OpenMP) will locate libgomp from the gnu installation. We would prefer to use libomp from the ROCm install. Query amdclang to find LLVM_TARGET_TRIPLE and use as a suffix for find_library(omp). On the other hand, the user can choose to use hipcc, which allows find_package(OpenMP) to properly locate libomp inside ROCm llvm. To include legacy support, I left the hardcoded -L with use of HIP_CLANG_ROOT as a fallback. This would be much simpler if hipBLAS defaulted to hipcc as the cmake compiler.
estewart08
requested review from
amcamd,
TorreZuk,
mahmoodw,
daineAMD,
bragadeesh,
NaveenElumalaiAMD and
rkamd
as code owners
| if (TARGET OpenMP::OpenMP_CXX) | ||
| set( COMMON_LINK_LIBS "OpenMP::OpenMP_CXX") |
There was a problem hiding this comment.
So aren't we missing the OpenMP::OpenMP_CXX link request on CUDA backend?
There was a problem hiding this comment.
True, I can move set( COMMON_LINK_LIBS ${LIBOMP_FOUND}) outside of the amd guard.
| @@ -116,13 +116,41 @@ if( BUILD_CLIENTS_BENCHMARKS OR BUILD_CLIENTS_TESTS) | |||
|
|
|||
| # if it fails to find OpenMP compile and link flags in strange configurations it can just use non-parallel reference computation | |||
| # if there is no omp.h to find the client compilation will fail and this should be obvious, used to be REQUIRED | |||
| find_package(OpenMP) | |||
| if( CMAKE_CXX_COMPILER MATCHES ".*/hipcc$" ) | |||
| find_package(OpenMP) | |||
There was a problem hiding this comment.
Similar question we shouldn't require NON hipcc to find_package(OpenMP) so can this just be moved two lines above? Sorry I meant other than hipcc we still want find_package to set cmake target etc.
There was a problem hiding this comment.
When the cmake compiler is hipcc find_package finds the correct ROCm libomp.so. Otherwise, gcc will get libgomp.so from gcc installation. I would rather get the libomp from ROCm to be consistent. With your current configuration cmake adds libgomp.so with -L/rpaths -lomp to the ROCm library. So I am fairly certain libgomp is ignored.
Remove amd check for rpath addtion. If we use the libomp.so from ROCm we need the rpath.
|
Hi @estewart08 are you still active on this? |
|
I am going to pick this effort back up, so I would prefer it to stay open for now. |
An upstream llvm change enables
LLVM_ENABLE_PER_TARGET_RUNTIME_DIR by default for the openmp build. This installs the openmp libraries into
/opt/rocm-ver/llvm/lib/x86_64-unknown-linux-gnuinstead of/opt/rocm-ver/llvm/lib. Currenty, hipBLAS only uses /lib.Since hipBLAS uses gcc by default,
find_package(OpenMP)will locate libgomp from the gnu installation. We would prefer to use libomp from the ROCm install. Query amdclang to find LLVM_TARGET_TRIPLE and use as a suffix forfind_library(omp).On the other hand, the user can choose to use hipcc, which allows
find_package(OpenMP)to properly locate libomp inside ROCm llvm.To include legacy support, I left the hardcoded
-L HIP_CLANG_ROOTas a fallback.This would be much simpler if hipBLAS defaulted to hipcc as the cmake compiler.
Summary of proposed changes:
find_library(omp).find_package(OpenMP).-L HIP_CLANG_ROOT/lib.find_package(LLVM)due torocm-llvm-dev/develnot being installed by default on various operating systems, which would not have the cmake config files.