Inconsistent handling of -0.0 between CPU and CUDA for many operators #162235
Description
🐛 Describe the bug
Similar to #114569, there are additional operators that show inconsistent handling of negative zero (-0.0) between CPU and CUDA.
On CPU these ops often preserve the -0.0 sign bit, while on CUDA they normalize it to +0.0 or change ordering in comparisons.
import torch
import torch.nn.functional as F
x_single = torch.tensor([-0.0], dtype=torch.float32)
x_pair = torch.tensor([-0.0, 0.0], dtype=torch.float32)
print("=== CPU ===")
print("maximum:", torch.maximum(x_single, torch.tensor([0.0])))
print("relu:", F.relu(x_single))
print("relu6:", F.relu6(x_single))
print("hardtanh:", F.hardtanh(x_single, 0.0, 1.0))
print("argsort:", torch.argsort(x_pair))
vals_cpu, idx_cpu = torch.sort(x_pair)
print("sort:", vals_cpu, idx_cpu)
print("amin:", torch.amin(x_pair, dim=0))
print("amax:", torch.amax(x_pair, dim=0))
device = torch.device("cuda")
x_single_cuda = x_single.to(device)
x_pair_cuda = x_pair.to(device)
print("\n=== GPU ===")
print("maximum:", torch.maximum(x_single_cuda, torch.tensor([0.0], device=device)))
print("relu:", F.relu(x_single_cuda))
print("relu6:", F.relu6(x_single_cuda))
print("hardtanh:", F.hardtanh(x_single_cuda, 0.0, 1.0))
print("argsort:", torch.argsort(x_pair_cuda))
vals_cuda, idx_cuda = torch.sort(x_pair_cuda)
print("sort:", vals_cuda, idx_cuda)
print("amin:", torch.amin(x_pair_cuda, dim=0))
print("amax:", torch.amax(x_pair_cuda, dim=0))Output:
=== CPU ===
maximum: tensor([-0.])
relu: tensor([-0.])
relu6: tensor([-0.])
hardtanh: tensor([-0.])
argsort: tensor([0, 1])
sort: tensor([-0., 0.]) tensor([0, 1])
amin: tensor(-0.)
amax: tensor(-0.)
=== GPU ===
maximum: tensor([0.], device='cuda:0')
relu: tensor([0.], device='cuda:0')
relu6: tensor([0.], device='cuda:0')
hardtanh: tensor([0.], device='cuda:0')
argsort: tensor([1, 0], device='cuda:0')
sort: tensor([0., -0.], device='cuda:0') tensor([1, 0], device='cuda:0')
amin: tensor(0., device='cuda:0')
amax: tensor(0., device='cuda:0')
Versions
PyTorch version: 2.5.1+cu124
Is debug build: False
CUDA used to build PyTorch: 12.4
ROCM used to build PyTorch: N/A
OS: Ubuntu 22.04.4 LTS (x86_64)
GCC version: (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
Clang version: 14.0.0-1ubuntu1.1
CMake version: version 3.31.6
Libc version: glibc-2.35