Skip to content

Conversation

@wavefunction91
Copy link

Fix various CMake bugs stemming from ENABLE_CUDA

  1. LibRETT used to fail with parallel build + GNU Makefiles (ala the Umpire build bug and fix). This fix follows the prescription used in Fix parallel compilation on Umpire #340.
  2. Unsure how this ever worked, but for some reason (at least for CMake 3.25+), you need to explicitly link to the CUDA::cudart TARGET, because apparently it doesn't get propagated via other dependencies... Unclear why CUDA::cublas doesn't propagate it, but who knows. Before this fix, I get the following on my system (CUDA 11.17 CMake 3.25) when building and linking ta_test
[ 88%] Linking CXX executable ta_test
cd /home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/tests && /home/dbwy/Software/BuildSystems/CMake/3.25.0/install/bin/cmake -E cmake_link_script CMakeFiles/ta_test.dir/link.txt --verbose=1
/usr/bin/c++ -g -Wl,-rpath -Wl,/home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/objdir/../install/lib -Wl,--enable-new-dtags CMakeFiles/ta_test.dir/ta_test.cpp.o CMakeFiles/ta_test.dir/range1.cpp.o CMakeFiles/ta_test.dir/range.cpp.o CMakeFiles/ta_test.dir/type_traits.cpp.o CMakeFiles/ta_test.dir/tensor.cpp.o CMakeFiles/ta_test.dir/tensor_of_tensor.cpp.o CMakeFiles/ta_test.dir/tensor_tensor_view.cpp.o CMakeFiles/ta_test.dir/tensor_shift_wrapper.cpp.o CMakeFiles/ta_test.dir/tiled_range1.cpp.o CMakeFiles/ta_test.dir/tiled_range.cpp.o CMakeFiles/ta_test.dir/blocked_pmap.cpp.o CMakeFiles/ta_test.dir/round_robin_pmap.cpp.o CMakeFiles/ta_test.dir/hash_pmap.cpp.o CMakeFiles/ta_test.dir/cyclic_pmap.cpp.o CMakeFiles/ta_test.dir/replicated_pmap.cpp.o CMakeFiles/ta_test.dir/dense_shape.cpp.o CMakeFiles/ta_test.dir/sparse_shape.cpp.o CMakeFiles/ta_test.dir/distributed_storage.cpp.o CMakeFiles/ta_test.dir/tensor_impl.cpp.o CMakeFiles/ta_test.dir/array_impl.cpp.o CMakeFiles/ta_test.dir/index_list.cpp.o CMakeFiles/ta_test.dir/bipartite_index_list.cpp.o CMakeFiles/ta_test.dir/dist_array.cpp.o CMakeFiles/ta_test.dir/conversions.cpp.o CMakeFiles/ta_test.dir/eigen.cpp.o CMakeFiles/ta_test.dir/dist_op_dist_cache.cpp.o CMakeFiles/ta_test.dir/dist_op_group.cpp.o CMakeFiles/ta_test.dir/dist_op_communicator.cpp.o CMakeFiles/ta_test.dir/tile_op_noop.cpp.o CMakeFiles/ta_test.dir/tile_op_scal.cpp.o CMakeFiles/ta_test.dir/dist_eval_array_eval.cpp.o CMakeFiles/ta_test.dir/dist_eval_unary_eval.cpp.o CMakeFiles/ta_test.dir/tile_op_add.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_add.cpp.o CMakeFiles/ta_test.dir/tile_op_subt.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_subt.cpp.o CMakeFiles/ta_test.dir/dist_eval_binary_eval.cpp.o CMakeFiles/ta_test.dir/tile_op_mult.cpp.o CMakeFiles/ta_test.dir/tile_op_scal_mult.cpp.o CMakeFiles/ta_test.dir/tile_op_contract_reduce.cpp.o CMakeFiles/ta_test.dir/reduce_task.cpp.o CMakeFiles/ta_test.dir/proc_grid.cpp.o CMakeFiles/ta_test.dir/dist_eval_contraction_eval.cpp.o CMakeFiles/ta_test.dir/expressions.cpp.o CMakeFiles/ta_test.dir/expressions_sparse.cpp.o CMakeFiles/ta_test.dir/expressions_complex.cpp.o CMakeFiles/ta_test.dir/expressions_btas.cpp.o CMakeFiles/ta_test.dir/expressions_mixed.cpp.o CMakeFiles/ta_test.dir/foreach.cpp.o CMakeFiles/ta_test.dir/solvers.cpp.o CMakeFiles/ta_test.dir/initializer_list.cpp.o CMakeFiles/ta_test.dir/diagonal_array.cpp.o CMakeFiles/ta_test.dir/retile.cpp.o CMakeFiles/ta_test.dir/tot_dist_array_part1.cpp.o CMakeFiles/ta_test.dir/tot_dist_array_part2.cpp.o CMakeFiles/ta_test.dir/random.cpp.o CMakeFiles/ta_test.dir/trace.cpp.o CMakeFiles/ta_test.dir/tot_expressions.cpp.o CMakeFiles/ta_test.dir/annotation.cpp.o CMakeFiles/ta_test.dir/contraction_helpers.cpp.o CMakeFiles/ta_test.dir/s_t_t_contract_.cpp.o CMakeFiles/ta_test.dir/t_t_t_contract_.cpp.o CMakeFiles/ta_test.dir/t_s_t_contract_.cpp.o CMakeFiles/ta_test.dir/einsum.cpp.o CMakeFiles/ta_test.dir/linalg.cpp.o CMakeFiles/ta_test.dir/cp.cpp.o CMakeFiles/ta_test.dir/librett.cpp.o CMakeFiles/ta_test.dir/expressions_cuda_um.cpp.o CMakeFiles/ta_test.dir/tensor_um.cpp.o -o ta_test   -L/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/umpire-build/lib  -L/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/librett-build/src  -Wl,-rpath,/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/src:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/madness-build/src/madness/world:/home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/lapackpp-build:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/blaspp-build:/home/dbwy/Software/Boost/gcc/9.4.0/1.80.0/install/lib:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/umpire-build/lib:/usr/local/cuda-11.7/lib64:/home/dbwy/Development/NWChemEx/nwx_update_ta/build_ta/_deps/librett-build/src:/home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-libflame/lib/LP64:/home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-blis/lib/LP64 ../src/libtiledarray.so ../_deps/madness-build/src/madness/world/libMADworld.so /home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib/libmpicxx.so /home/dbwy/Software/MPI/MPICH/gcc/9.4.0/4.0.2/install/lib/libmpi.so -lpthread ../_deps/lapackpp-build/liblapackpp.so ../_deps/blaspp-build/libblaspp.so /usr/lib/gcc/x86_64-linux-gnu/9/libgomp.so /usr/lib/x86_64-linux-gnu/libpthread.so /home/dbwy/Software/Boost/gcc/9.4.0/1.80.0/install/lib/libboost_serialization.so.1.80.0 -lumpire /usr/local/cuda-11.7/lib64/libcublas.so /usr/local/cuda-11.7/lib64/libcublasLt.so /usr/local/cuda-11.7/lib64/libculibos.a /usr/local/cuda-11.7/lib64/libnvToolsExt.so -lrett /home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-libflame/lib/LP64/libflame.so /home/dbwy/Software/LinearAlgebra/AOCL/gcc/amd-blis/lib/LP64/libblis.so -lm 
/usr/bin/ld: CMakeFiles/ta_test.dir/dist_array.cpp.o: in function `array_suite::serialization_by_tile::test_method()':
/home/dbwy/Development/NWChemEx/nwx_update_ta/tiledarray/tests/dist_array.cpp:652: warning: the use of `mktemp' is dangerous, better use `mkstemp' or `mkdtemp'
/usr/bin/ld: CMakeFiles/ta_test.dir/expressions_cuda_um.cpp.o: undefined reference to symbol 'cudaStreamSynchronize@@libcudart.so.11.0'

@wavefunction91 wavefunction91 requested a review from evaleev August 18, 2023 16:27
@evaleev evaleev merged commit efba9d1 into master Aug 21, 2023
@evaleev evaleev deleted the dbwy/bugfix/cmake_cuda branch August 21, 2023 13:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants