Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix dedup of linker flags for shared libs on cmake <= 3.12 #3176

Merged
merged 1 commit into from
Jul 10, 2020

Conversation

jjwilke
Copy link

@jjwilke jjwilke commented Jul 9, 2020

Testing is currently missing issues with shared library builds.
Because CMake <= 3.12 does not provide a 'target_link_options', you have to hack link options through target_link_libraries. These do not get effectively deduplicated. This leads to the same flag appearing multiple times on the link lines in certain cases. For the -arch=X flag with nvcc, this causes an actual error.

This only adds flags to kokkoscore avoiding the need for deduplication.

@dalg24 dalg24 requested a review from jrmadsen July 9, 2020 19:37
@masterleinad
Copy link
Contributor

Can you describe a situation where we run into problems, i.e., warnings or errors? Building Kokkos as a shared library, I see

/tmp/kokkos/bin/nvcc_wrapper  -O3   CMakeFiles/example.dir/cmake_example.cpp.o CMakeFiles/example.dir/foo.f.o  -o example -Wl,-rpath,/tmp/kokkos-install-with-warnings/lib /tmp/kokkos-install-with-warnings/lib/libkokkoscontainers.so.3.1.1 /tmp/kokkos-install-with-warnings/lib/libkokkoscore.so.3.1.1 /usr/lib/x86_64-linux-gnu/libcuda.so /tmp/cuda-11/lib64/libcudart.so /usr/lib/x86_64-linux-gnu/libdl.so -Wall -Wunused-parameter -Wshadow -pedantic -Wtype-limits -Wuninitialized -arch=sm_70 -lgfortran -lquadmath 

in the link step (using CMake 3.10.2).

@jjwilke
Copy link
Author

jjwilke commented Jul 9, 2020

This happens in the build for kokkoscontainers. It gets double -arch flags. One set comes transitively from kokkoscore and the other gets added directly to kokkoscontainers.

@masterleinad
Copy link
Contributor

This happens in the build for kokkoscontainers. It gets double -arch flags. One set comes transitively from kokkoscore and the other gets added directly to kokkoscontainers.

Su you get errors when building Kokkos? What are the CMake parameters for the configuration that is failing for you?

@jjwilke
Copy link
Author

jjwilke commented Jul 9, 2020

cmake \
  -DCMAKE_CXX_COMPILER=`pwd`/../bin/nvcc_wrapper \
  -DKokkos_ENABLE_CUDA=ON \
  -DBUILD_SHARED_LIBS=ON \
  ..
>cmake --version
cmake version 3.12.2
>nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Sun_Jul_28_19:07:16_PDT_2019
Cuda compilation tools, release 10.1, V10.1.243

And the final output is:

[ 95%] Linking CXX shared library libkokkoscontainers.so
cd /ascldap/users/jjwilke/Programs/kokkos/develop/jenkins-build/containers/src && /net/watson.sandia.gov/storage/fast/projects/sems/install/rhel7-x86_64/sems/utility/cmake/3.12.2/bin/cmake -E cmake_link_script CMakeFiles/kokkoscontainers.dir/link.txt --verbose=1
/ascldap/users/jjwilke/Programs/kokkos/develop/jenkins-build/../bin/nvcc_wrapper -fPIC -O2 -g -DNDEBUG  -shared -Wl,-soname,libkokkoscontainers.so.3.1.1 -o libkokkoscontainers.so.3.1.1 CMakeFiles/kokkoscontainers.dir/impl/Kokkos_UnorderedMap_impl.cpp.o -Wl,-rpath,/ascldap/users/jjwilke/Programs/kokkos/develop/jenkins-build/core/src: -arch=sm_70 ../../core/src/libkokkoscore.so.3.1.1 -arch=sm_70 /usr/lib64/libcuda.so /projects/sems/install/rhel7-x86_64/sems/compiler/cuda/10.1/base/lib64/libcudart.so /usr/lib64/libdl.so 
nvcc fatal   : redefinition of argument 'gpu-architecture'

@masterleinad
Copy link
Contributor

OK, I am seeing

cd /tmp/kokkos_new/build_jeremy/containers/src && /usr/bin/cmake -E cmake_link_script CMakeFiles/kokkoscontainers.dir/link.txt --verbose=1
/tmp/kokkos_new/build_jeremy/../bin/nvcc_wrapper -fPIC -O2 -g -DNDEBUG  -shared -Wl,-soname,libkokkoscontainers.so.3.1.1 -o libkokkoscontainers.so.3.1.1 CMakeFiles/kokkoscontainers.dir/impl/Kokkos_UnorderedMap_impl.cpp.o -Wl,-rpath,/tmp/kokkos_new/build_jeremy/core/src: -arch=sm_70 ../../core/src/libkokkoscore.so.3.1.1 -arch=sm_70 /usr/lib/x86_64-linux-gnu/libcuda.so /tmp/cuda-11/lib64/libcudart.so /usr/lib/x86_64-linux-gnu/libdl.so 

but no error with CUDA11 or CUDA9 and I can compile a dummy project with the options you posted.
With this patch, I am seeing

cd /tmp/kokkos_new/build_jeremy/containers/src && /tmp/Sources/cmake-3.12.0-Linux-x86_64/bin/cmake -E cmake_link_script CMakeFiles/kokkoscontainers.dir/link.txt --verbose=1
/tmp/kokkos_new/build_jeremy/../bin/nvcc_wrapper -fPIC -O2 -g -DNDEBUG  -shared -Wl,-soname,libkokkoscontainers.so.3.1.1 -o libkokkoscontainers.so.3.1.1 CMakeFiles/kokkoscontainers.dir/impl/Kokkos_UnorderedMap_impl.cpp.o -Wl,-rpath,/tmp/kokkos_new/build_jeremy/core/src: ../../core/src/libkokkoscore.so.3.1.1 -arch=sm_70 /usr/lib/x86_64-linux-gnu/libcuda.so /tmp/cuda-11/lib64/libcudart.so /usr/lib/x86_64-linux-gnu/libdl.so 

As expected they only differ by the duplicated -arch=sm_70 flag and I can also compiler a dummy project.

Even though I don't see the error with a current version, I am fine with this pull request.

@dalg24 dalg24 merged commit b0efbe3 into kokkos:develop Jul 10, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants