[CUDA][HIP] std::min/max differs from libstdc++/libc++ about NaN

[yxsamliu]

CUDA/HIP defines overloaded version of host device std::min/max with higher precedence than those in standard C++ headers:

llvm-project/clang/lib/Headers/cuda_wrappers/algorithm

Line 101 in 6ee845d

min(const __T &__a, const __T &__b) {

https://github.com/gcc-mirror/gcc/blob/d9c90c82d900fdae95df4499bf5f0a4ecb903b53/libstdc%2B%2B-v3/include/bits/stl_algobase.h#L233

llvm-project/libcxx/include/__algorithm/min.h

Line 36 in 6ee845d

return std::min(__a, __b, __less<>());

However, there is a subtle difference of the CUDA/HIP version compared to libstdc++/libc++ version.

CUDA/HIP version is like __a < __b ? __a : __b. If there is NaN, it chooese __b.

libstdc++/libc++ version is like __b < __a ? __b : __a. If there is NaN, it chooses __a.

This difference causes annoyance for PyTorch

ROCm/HIP#2209 (comment)

according to https://en.cppreference.com/w/cpp/algorithm/min

std::min(a,b) returns the smaller of a and b. If the values are equivalent, returns a. It seems libstdc++/libc++ behavior is correct. CUDA/HIP std::min/max needs to be fixed.

@Artem-B