[Build] build errors with 1.15.1 and CUDA enabled

[pfeatherstone]

Describe the issue

I'm building onnxruntime from source with CUDA enabled. I get the same build error with 1.15.1 and 1.15.0. I haven't tried earlier versions.

Urgency

ASAP

Target platform

Linux

Build script

wget https://github.com/microsoft/onnxruntime/archive/refs/tags/v1.15.1.tar.gz
tar -zxvf v.1.15.1.tar.gz
cd onnxruntime-1.15.1
./build.sh --config Release --parallel  --cmake_generator Ninja --build_shared_lib --use_cuda --cuda_home /usr/local/cuda --cudnn_home /usr/local/cuda --skip_submodule_sync

Error / output

onnxruntime/core/providers/cuda/cu_inc/common.cuh(80): error: more than one operator "+" matches these operands:
            function "operator+(const __half &, const __half &)" (declared at line 606 of /usr/local/cuda/include/cuda_fp16.hpp)
            function "onnxruntime::cuda::operator+(const half &, const half &)" (declared at line 23)
            operand types are: const __half + const __half
    r.x = lh.x + rh.x;

Visual Studio Version

August 2023 (version 1.82)

GCC / Compiler Version

11.4.0

NVCC Version

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2023 NVIDIA Corporation
Built on Tue_Aug_15_22:02:13_PDT_2023
Cuda compilation tools, release 12.2, V12.2.140
Build cuda_12.2.r12.2/compiler.33191640_0

CUDA version

NVIDIA-SMI 535.104.05             Driver Version: 535.104.05   CUDA Version: 12.2

[snnn]

You need to use the latest main branch. The last release was published before CUDA 12.2 was out so we couldn't test the compatibility at the time when we published it.

[pfeatherstone]

So use master ? Is that stable?

[snnn]

Or you downgrade the CUDA version to 11.8, or another CUDA version that works.

[pfeatherstone]

OK I will downgrade to 11.8. To be honest, pytorch uses 11.8 as well so it's probs best I use whatever onnxruntime and torch use (even though pytorch ships with cuda as well)

[pfeatherstone]

Will the next release of Ort work with cuda 12 ?

[snnn]

If you build the next release from source, yes.

[pfeatherstone]

Cool. When will the next release be ?

[snnn]

In a few weeks.

[agirault]

For reference: fixed in cee050f (part #16831, merged as 7b9d1f1 and included since release v1.16.0)