torch.rand can sample the upper bound for lower precision floating point dtypes on CUDA

[pmeier]

Documentation:

Returns a tensor filled with random numbers from a uniform distribution on the [half-open] interval [0,1)

import itertools

import torch

for device, dtype in itertools.product(
    ["cpu", "cuda"],
    [
        torch.float16,
        torch.bfloat16,
        torch.float32,
        torch.float64,
        torch.complex32,
        torch.complex64,
        torch.complex128,
    ],
):
    torch.manual_seed(0)
    # I only used this high number of samples to make sure the other dtypes are not affected
    # On my machine 1_000 was sufficient for the check to fail for bfloat16, 
    # and 10_000 for float16 and complex32
    t = torch.rand(10_000_000, dtype=dtype, device=device)
    if dtype.is_complex:
        t = torch.view_as_real(t)

    print(f"{dtype}, {device}: {'PASS' if (t != 1).all() else 'FAIL'}")

torch.float16, cpu: PASS
torch.bfloat16, cpu: PASS
torch.float32, cpu: PASS
torch.float64, cpu: PASS
torch.complex32, cpu: PASS
torch.complex64, cpu: PASS
torch.complex128, cpu: PASS
torch.float16, cuda: FAIL
torch.bfloat16, cuda: FAIL
torch.float32, cuda: PASS
torch.float64, cuda: PASS
torch.complex32, cuda: FAIL
torch.complex64, cuda: PASS
torch.complex128, cuda: PASS

Failures happen for float16, bfloat16, and complex32 only on CUDA.

This was detected in #96331, which uses Tensor.uniform_ under the hood, but I guess internally it is the same kernel.

cc @ezyang @gchanan @zou3519 @pbelevich