linux build from source error: no instance of overloaded function "atomicAdd" matches the argument list

[qizhen816]

Describe the bug
/onnxruntime/onnxruntime/core/providers/cuda/atomic/common.cuh(67): error: no instance of overloaded function "atomicAdd" matches the argument list
argument types are: (half *, half)

1 error detected in the compilation of "/tmp/tmpxft_0002f95b_00000000-9_reduction_functions.compute_70.cpp1.ii".
CMakeFiles/onnxruntime_providers_cuda.dir/build.make:1144: recipe for target 'CMakeFiles/onnxruntime_providers_cuda.dir/mnt/onnxruntime/onnxruntime/core/providers/cuda/reduction/reduction_functions.cu.o' failed

Urgency
If there are particular important use cases blocked by this or strict project-related timelines, please share more information and dates. If there are no hard deadlines, please specify none.

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 16.04
• ONNX Runtime installed from (source or binary):source
• ONNX Runtime version:1.3.0 master
• Python version:
• Visual Studio version (if applicable):
• GCC/Compiler version (if compiling from source): gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.12)
• CUDA/cuDNN version: 9.2
• GPU model and memory: GTX1080Ti

To Reproduce
sh ./build.sh --use_cuda --cuda_version 9.2 --cudnn_home /usr/local/cuda-9.2 --cuda_home /usr/local/cuda-9.2

[ 30%] Building CUDA object CMakeFiles/onnxruntime_providers_cuda.dir/mnt/onnxruntime/onnxruntime/core/providers/cuda/reduction/reduction_functions.cu.o
/mnt/onnxruntime/onnxruntime/core/providers/cuda/atomic/common.cuh(67): error: no instance of overloaded function "atomicAdd" matches the argument list
            argument types are: (half *, half)

1 error detected in the compilation of "/tmp/tmpxft_0002f95b_00000000-9_reduction_functions.compute_70.cpp1.ii".
CMakeFiles/onnxruntime_providers_cuda.dir/build.make:1144: recipe for target 'CMakeFiles/onnxruntime_providers_cuda.dir/mnt/onnxruntime/onnxruntime/core/providers/cuda/reduction/reduction_functions.cu.o' failed
make[2]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/mnt/onnxruntime/onnxruntime/core/providers/cuda/reduction/reduction_functions.cu.o] Error 1
CMakeFiles/Makefile2:359: recipe for target 'CMakeFiles/onnxruntime_providers_cuda.dir/all' failed
make[1]: *** [CMakeFiles/onnxruntime_providers_cuda.dir/all] Error 2
Makefile:143: recipe for target 'all' failed
make: *** [all] Error 2
Traceback (most recent call last):
  File "/mnt/onnxruntime/tools/ci_build/build.py", line 1809, in <module>
    sys.exit(main())
  File "/mnt/onnxruntime/tools/ci_build/build.py", line 1705, in main
    build_targets(args, cmake_path, build_dir, configs, args.parallel)
  File "/mnt/onnxruntime/tools/ci_build/build.py", line 868, in build_targets
    run_subprocess(cmd_args, env=env)
  File "/mnt/onnxruntime/tools/ci_build/build.py", line 402, in run_subprocess
    env=my_env, shell=shell)
  File "/usr/lib/python3.6/subprocess.py", line 438, in run
    output=stdout, stderr=stderr)
subprocess.CalledProcessError: Command '['/usr/local/bin/cmake', '--build', '/mnt/onnxruntime/build/Linux/Debug', '--config', 'Debug']' returned non-zero exit status 2.

[yuslepukhin]

Although your device supports the functionality, it is not defined within CUDA 9.2 headers/libraries that you are using. To get compiler and library support for your device capabilities we recommend CUDA 10.2

[qizhen816]

Sadly, the device I use is running mutiple services which means I can't update nvidia driver and restart. CUDA 9.2 is the maximum support of the driver. Is there any way I can build onnxruntime using CUDA 9.2?

[yuslepukhin]

You can try modifying the source code in a way that the other branch of #ifdef is taken. But there might be more of such places. We do not build or test it though.

[qizhen816]

Is there any instruction of it? I wanna try.

[qizhen816]

changed 2 places in onnxruntime/core/providers/cuda/atomic/common.cuh, build passed.