LibTorch cannot be used without nvcc

[christian-rauch]

🐛 Describe the bug

When using LibTorch in CMake via find_package(Torch REQUIRED) it tries to find CUDA via nvcc. When only the runtime libraries are installed, e.g. sudo apt install cuda-libraries-11-3 then it will fail with:

CUDA_TOOLKIT_ROOT_DIR not found or specified
-- Could NOT find CUDA (missing: CUDA_TOOLKIT_ROOT_DIR CUDA_NVCC_EXECUTABLE CUDA_INCLUDE_DIRS CUDA_CUDART_LIBRARY) 
CMake Warning at [...]/cmake/Caffe2/public/cuda.cmake:31 (message):
  Caffe2: CUDA cannot be found.  Depending on whether you are building Caffe2
  or a Caffe2 dependent library, the next warning / error will give you more
  info.

Since LibTorch is already compiled, it should not require nvcc and other development libraries to be installed.

Looking at Caffe2/public/cuda.cmake and https://github.com/pytorch/pytorch/blob/v1.11.0/cmake/public/cuda.cmake#L29, it seems that the deprecated find_package(CUDA) is used. Instead, the new FindCUDAToolkit should be used. This will work without nvcc installed.

Versions

1.11 from https://download.pytorch.org/libtorch/cu113/libtorch-cxx11-abi-shared-with-deps-1.11.0%2Bcu113.zip

cc @malfet @seemethere

[christian-rauch]

Btw, why does CUDA actually have to be installed anyway? It appears to me that none of the CUDA shared objects are actually linked:

$ lddtree libtorch_cuda_cu.so
libtorch_cuda_cu.so => ./libtorch_cuda_cu.so (interpreter => none)
    libcudart-a7b20f20.so.11.0 => ./libcudart-a7b20f20.so.11.0
    libtorch_cuda_cpp.so => ./libtorch_cuda_cpp.so
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
    libc10_cuda.so => ./libc10_cuda.so
    libnvToolsExt-24de1d56.so.1 => ./libnvToolsExt-24de1d56.so.1
    libc10.so => ./libc10.so
    libtorch_cpu.so => ./libtorch_cpu.so
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1
    libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6
    libgomp-52f2fd74.so.1 => ./libgomp-52f2fd74.so.1
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6
    ld-linux-x86-64.so.2 => /lib/x86_64-linux-gnu/ld-linux-x86-64.so.2

[xsacha]

I've been manually modifying the CMake files to remove references to CUDA and CUDNN because they should only be required for the build process. Anything that consumes the resulting pytorch library shouldn't care about development files for CUDA/CUDNN.
All we need for the inclusion process (CMake) of libtorch should be any shared libraries (if shared was selected) for the distribution.

So yes, very valid! It would be great if this were fixed.