Building C++ API from source failed

[juntingzh]

I have CUDA 11.2 and CUDNN 8.2.1, PyTorch 1.10 and when I try to build pytorch_scatter from source I hit this error:

[  9%] Building CXX object CMakeFiles/torchscatter.dir/csrc/cpu/scatter_cpu.cpp.o
[ 18%] Building CXX object CMakeFiles/torchscatter.dir/csrc/cpu/segment_coo_cpu.cpp.o
[ 27%] Building CXX object CMakeFiles/torchscatter.dir/csrc/cpu/segment_csr_cpu.cpp.o
[ 36%] Building CUDA object CMakeFiles/torchscatter.dir/csrc/cuda/scatter_cuda.cu.o
/home/junting_zhang/projects/pytorch_scatter/csrc/cuda/utils.cuh(12): error: more than one user-defined conversion from "const c10::Half" to "__half" applies:
            function "__half::__half(float)"
            function "__half::__half(__half &&)"

/home/junting_zhang/projects/pytorch_scatter/csrc/cuda/utils.cuh(18): error: more than one user-defined conversion from "const c10::Half" to "__half" applies:
            function "__half::__half(float)"
            function "__half::__half(__half &&)"

2 errors detected in the compilation of "/home/junting_zhang/projects/pytorch_scatter/csrc/cuda/scatter_cuda.cu".
CMakeFiles/torchscatter.dir/build.make:120: recipe for target 'CMakeFiles/torchscatter.dir/csrc/cuda/scatter_cuda.cu.o' failed
make[2]: *** [CMakeFiles/torchscatter.dir/csrc/cuda/scatter_cuda.cu.o] Error 1
CMakeFiles/Makefile2:94: recipe for target 'CMakeFiles/torchscatter.dir/all' failed
make[1]: *** [CMakeFiles/torchscatter.dir/all] Error 2
Makefile:148: recipe for target 'all' failed
make: *** [all] Error 2

Any idea on how to fix?

[rusty1s]

Are you using PyTorch nightly? There might be some changes in nightly PyTorch internals that prevent us from building external packages. Those get usually fixed once PyTorch 1.10 is fully released.

[kotatsuyaki]

This is probably not caused by the usage of PyTorch nightly though, I'm getting the same error with libtorch 1.9.0+cu111. The error also happens with torch_sparse >= 0.6.11, which was the release that introduced half-precision support.

I'm temporarily reverting to using release 2.0.7, because it builds without this issue.

Build log

Scanning dependencies of target torchscatter
[  9%] Building CXX object CMakeFiles/torchscatter.dir/csrc/cpu/scatter_cpu.cpp.o
[ 18%] Building CXX object CMakeFiles/torchscatter.dir/csrc/cpu/segment_coo_cpu.cpp.o
[ 27%] Building CXX object CMakeFiles/torchscatter.dir/csrc/cpu/segment_csr_cpu.cpp.o
[ 36%] Building CUDA object CMakeFiles/torchscatter.dir/csrc/cuda/segment_coo_cuda.cu.o
[ 45%] Building CUDA object CMakeFiles/torchscatter.dir/csrc/cuda/scatter_cuda.cu.o
[ 54%] Building CUDA object CMakeFiles/torchscatter.dir/csrc/cuda/segment_csr_cuda.cu.o
nvcc warning : The 'compute_35', 'compute_37', 'compute_50', 'sm_35', 'sm_37' and 'sm_50' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets
to suppress warning).
nvcc warning : The 'compute_35', 'compute_37', 'compute_50', 'sm_35', 'sm_37' and 'sm_50' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets
to suppress warning).
nvcc warning : The 'compute_35', 'compute_37', 'compute_50', 'sm_35', 'sm_37' and 'sm_50' architectures are deprecated, and may be removed in a future release (Use -Wno-deprecated-gpu-targets
to suppress warning).
/build/source/csrc/cuda/utils.cuh(12): error: more than one user-defined conversion from "const c10::Half" to "__half" applies:
            function "__half::__half(float)"
            function "__half::__half(__half &&)"

/build/source/csrc/cuda/utils.cuh(12): error: more than one user-defined conversion from "const c10::Half" to "__half" applies:
            function "__half::__half(float)"
            function "__half::__half(__half &&)"

/build/source/csrc/cuda/utils.cuh(18): error: more than one user-defined conversion from "const c10::Half" to "__half" applies:
            function "__half::__half(float)"
            function "__half::__half(__half &&)"

/build/source/csrc/cuda/utils.cuh(18): error: more than one user-defined conversion from "const c10::Half" to "__half" applies:
            function "__half::__half(float)"
            function "__half::__half(__half &&)"

/build/source/csrc/cuda/utils.cuh(12): error: more than one user-defined conversion from "const c10::Half" to "__half" applies:
            function "__half::__half(float)"
            function "__half::__half(__half &&)"

/build/source/csrc/cuda/utils.cuh(18): error: more than one user-defined conversion from "const c10::Half" to "__half" applies:
            function "__half::__half(float)"
            function "__half::__half(__half &&)"

2 errors detected in the compilation of "/build/source/csrc/cuda/scatter_cuda.cu".
2 errors detected in the compilation of "/build/source/csrc/cuda/segment_csr_cuda.cu".

[rusty1s]

Thanks. Which GPU are you using? Can you also check if specifying a compiler flag helps to resolve this?

export TORCH_CUDA_ARCH_LIST = "7.5;8.0;8.6"

[kotatsuyaki]

It's an RTX3070. I'll try the environment variable out and report again.

[akshitgandhi]

I am also facing the same issue. Did anyone figure out a solution to this problem? I also tried latest pytorch_scatter release=2.0.9 and libtorch=1.10 with Cuda 11.1 and gcc 10, but it still fails with this same error.

[alexnikulkov]

Are there any updates here? Do we think the half-precision operations were implemented robustly or could it make sense to back them out if they are the source of error?

[rusty1s]

I'm still not sure what's causing this TBH, and half precision support should work fine when using our provided wheels. One option would be to add a USE_HALF flag that users that want to install from source and encounter this error can use.

[alepack89]

Hi

I don't know if this may help, but I think that the problem may be that there are two possible conversion methods from at::Half to __half.
__half can be created either from a float or a __half and since at::Half has a cast-to-float operator defined, the conversion results ambiguous

If I call the correct conversion operator explicitly the code can be built without errors

#pragma once

#include <torch/extension.h>

#define CHECK_CUDA(x)                                                          \
  AT_ASSERTM(x.device().is_cuda(), #x " must be CUDA tensor")
#define CHECK_INPUT(x) AT_ASSERTM(x, "Input mismatch")

__device__ __inline__ at::Half __shfl_up_sync(const unsigned mask,
                                              const at::Half var,
                                              const unsigned int delta) {
  return __shfl_up_sync(mask, var.operator __half(), delta);
}

__device__ __inline__ at::Half __shfl_down_sync(const unsigned mask,
                                                const at::Half var,
                                                const unsigned int delta) {
  return __shfl_down_sync(mask, var.operator __half(), delta);
}
``
Could this be the solution?

[rusty1s]

Thanks for sharing this trick, integrated via #280. I am wondering why this only results in an error when building with CMake though.

[alepack89]

Hi

Checking the make commands that are issued, I think that the CMake version is missing the following definitions

add_definitions(-D__CUDA_NO_HALF_CONVERSIONS__)
add_definitions(-D__CUDA_NO_BFLOAT16_CONVERSIONS__)
add_definitions(-D__CUDA_NO_HALF2_OPERATORS__)

Adding them results in a correct build even with the original code

[alepack89]

Hi

I think that the same problem can be found in the pytorch_sparse project.
The file is
csrc/cuda/cuda_utils.cuh

Is this the case?

Thank you