Failed to compute shorthash for libnvrtc.so

[judicaelclair]

With latest main, I get warning

CMake Warning at [..]/torch/share/cmake/Caffe2/public/cuda.cmake:150 (message):
  Failed to compute shorthash for libnvrtc.so

I believe this bug was introduced by 479ce5e as cmake now attempts to use the Python::Interpreter target, which cannot be resolved within an execute_process context. Should be reverted back to:

  if("${Python_EXECUTABLE}" STREQUAL "")
    set(_python_exe "python3")
  else()
    set(_python_exe "${Python_EXECUTABLE}")
  endif()
  execute_process(
      COMMAND "${_python_exe}" -c

Alternatively, the python executable path should be manually extracted from Python::Interpreter.

Versions

Torch version db4c7bb

cc @malfet @seemethere

[malfet]

@judicaelclair does it happen when you are building extensions, right?

[malfet]

Fixed by #130651 but if the problem reappears, please do not hesitate to comment here or open a new issue.

[Fr4nk1inCs]

Getting the same warning with torch-2.5.1.

The _retval in

pytorch/cmake/public/cuda.cmake

Lines 133 to 138 in a8d6afb

	find_package(Python COMPONENTS Interpreter)
	execute_process(
	COMMAND Python::Interpreter -c
	"import hashlib;hash=hashlib.sha256();hash.update(open('${CUDA_NVRTC_LIB}','rb').read());print(hash.hexdigest()[:8])"
	RESULT_VARIABLE _retval
	OUTPUT_VARIABLE CUDA_NVRTC_SHORTHASH)

is no such file or directory. But running the same python commands successfully output the hash digest.

[zhanwenchen]

@malfet @xuhancn Please re-open. The issue wasn't due to CUDA_NVRTC_LIB not existing, but Python::Interpreter not existing (I can't figure out why). Here are my debug outputs by printing:

# cuda.cmake
if(CUDA_NVRTC_LIB AND NOT CUDA_NVRTC_SHORTHASH)
  find_package(Python COMPONENTS Interpreter REQUIRED)
  execute_process(
    COMMAND ${Python_EXECUTABLE} -c
    # "import hashlib;hash=hashlib.sha256();print(f'CUDA_NVRTC_LIB=${CUDA_NVRTC_LIB}')"
    "import hashlib;hash=hashlib.sha256();hash.update(open('${CUDA_NVRTC_LIB}','rb').read());print(hash.hexdigest()[:8])"
    RESULT_VARIABLE _retval
    OUTPUT_VARIABLE CUDA_NVRTC_SHORTHASH)
  message(STATUS "Python_EXECUTABLE=(${Python_EXECUTABLE}); Python_INTERPRETER=(${Python_INTERPRETER}); _retval=${_retval}. CUDA_NVRTC_LIB=${CUDA_NVRTC_LIB}")

  if(NOT _retval EQUAL 0)
.......

# output:
  -- Python_EXECUTABLE=(/miniconda3/bin/python); Python_INTERPRETER=(); _retval=0. CUDA_NVRTC_LIB=/usr/local/cuda-12.6/lib64/libnvrtc.so

After this fix, I was able to get

  -- /usr/local/cuda-12.6/lib64/libnvrtc.so shorthash is a1f1e7d6