max_unpool2d is not deterministic

[upupyy]

🐛 Describe the bug

max_unpool2d is not deterministic

import torch

def test():
    kernel_size = 2
    stride = 2
    unpool = torch.nn.MaxUnpool2d(kernel_size, stride=stride, )

    input0_tensor = torch.rand([1, 1, 2, 2], dtype=torch.float32)
    input1_tensor = torch.randint(-2, 8, [1, 1, 2, 2], dtype=torch.int64)

    input0 = input0_tensor.clone()
    input1 = input1_tensor.clone()
    try:
        res1 = unpool(input0, input1)
    except Exception:
        res1 = torch.tensor(0)

    input0 = input0_tensor.clone()
    input1 = input1_tensor.clone()
    try:
        res2 = unpool(input0, input1)
    except Exception:
        res2 = torch.tensor(0)

    if torch.allclose(res1, res2) == False:
        print(input0_tensor)
        print(input1_tensor)
        print(res1)
        print(res2)
        return True
    else:
        return False

for _ in range(1000):
    if test():
        break
# tensor([[[[0.8725, 0.3154],
#           [0.7132, 0.8304]]]])
# tensor([[[[1, 6],
#           [1, 5]]]])
# tensor([[[[0.0000, 0.7132, 0.0000, 0.0000],
#           [0.0000, 0.8304, 0.3154, 0.0000],
#           [0.0000, 0.0000, 0.0000, 0.0000],
#           [0.0000, 0.0000, 0.0000, 0.0000]]]])
# tensor([[[[0.0000, 0.8725, 0.0000, 0.0000],
#           [0.0000, 0.8304, 0.3154, 0.0000],
#           [0.0000, 0.0000, 0.0000, 0.0000],
#           [0.0000, 0.0000, 0.0000, 0.0000]]]])

Versions

pytorch: 1.11.0

cc @albanD @mruberry @jbschlosser @walterddr @kshitij12345 @kurtamohler

[mruberry]

@kurtamohler -- this is interesting -- I don't think we call out CPU max_unpool2d as nondeterministic?

[kurtamohler]

We don't have max_unpool2d on the list of nondeterministic operations. Looks like it is only nondeterministic on CPU, not CUDA, and if there are more than 2 threads, not if torch.set_num_threads(1) or torch.set_num_threads(2) is set (at least on my machine).

I can add a nondeterministic error to it

[kurtamohler]

MaxUnpool3d has similar behavior, so I'll also add an error to it