Gradient tests failing for `max_unpool2d`

[krshrimali]

🐛 Bug

While trying to add OpInfo for max_unpool2d, the following errors were raised:

Caution: long snippet alert!!

======================================================================
ERROR: test_fn_grad_nn_functional_max_unpool2d_cpu_float64 (__main__.TestGradientsCPU)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
    raise rte
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 371, in instantiated_test
    result = test(self, **param_kwargs)
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 737, in test_wrapper
    return test(*args, **kwargs)
  File "/home/krshrimali/pytorch/test/test_ops.py", line 603, in test_fn_grad
    self._grad_test_helper(device, dtype, op, op.get_op())
  File "/home/krshrimali/pytorch/test/test_ops.py", line 586, in _grad_test_helper
    return self._check_helper(device, dtype, op, variant, 'gradcheck', check_forward_ad=check_forward_ad,
  File "/home/krshrimali/pytorch/test/test_ops.py", line 559, in _check_helper
    self.assertTrue(gradcheck(fn, gradcheck_args,
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_utils.py", line 2718, in gradcheck
    return torch.autograd.gradcheck(fn, inputs, **kwargs)
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1273, in gradcheck
    return _gradcheck_helper(**args)
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1286, in _gradcheck_helper
    _gradcheck_real_imag(gradcheck_fn, func, func_out, tupled_inputs, outputs, eps,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 947, in _gradcheck_real_imag
    gradcheck_fn(func, func_out, tupled_inputs, outputs, eps,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1168, in _fast_gradcheck
    _check_analytical_numerical_equal(analytical_vJu, numerical_vJu, complex_indices,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1148, in _check_analytical_numerical_equal
    raise GradcheckError(_get_notallclose_msg(a, n, j, i, complex_indices, test_imag, is_forward_ad) + jacobians_str)
torch.autograd.gradcheck.GradcheckError: Jacobian mismatch for output 0 with respect to input 0,
numerical:tensor(0.2397)
analytical:tensor(0.8019)

The above quantities relating the numerical and analytical jacobians are computed
in fast mode. See: https://github.com/pytorch/pytorch/issues/53876 for more background
about fast mode. Below, we recompute numerical and analytical jacobians in slow mode:

Numerical:
 tensor([[1.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        ...,
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 1.0000]])
Analytical:
tensor([[1., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 1.,  ..., 0., 0., 0.],
        [0., 0., 1.,  ..., 0., 0., 0.],
        ...,
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 1.],
        [0., 0., 0.,  ..., 0., 0., 1.]])

The max per-element difference (slow mode) is: 1.0.


======================================================================
ERROR: test_fn_gradgrad_nn_functional_max_unpool2d_cpu_float64 (__main__.TestGradientsCPU)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
    raise rte
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 371, in instantiated_test
    result = test(self, **param_kwargs)
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 737, in test_wrapper
    return test(*args, **kwargs)
  File "/home/krshrimali/pytorch/test/test_ops.py", line 625, in test_fn_gradgrad
    self._gradgrad_test_helper(device, dtype, op, op.get_op())
  File "/home/krshrimali/pytorch/test/test_ops.py", line 591, in _gradgrad_test_helper
    return self._check_helper(device, dtype, op, variant, 'gradgradcheck')
  File "/home/krshrimali/pytorch/test/test_ops.py", line 569, in _check_helper
    self.assertTrue(gradgradcheck(fn, gradcheck_args,
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_utils.py", line 2738, in gradgradcheck
    return torch.autograd.gradgradcheck(fn, inputs, grad_outputs, **kwargs)
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1402, in gradgradcheck
    return gradcheck(
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1273, in gradcheck
    return _gradcheck_helper(**args)
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1286, in _gradcheck_helper
    _gradcheck_real_imag(gradcheck_fn, func, func_out, tupled_inputs, outputs, eps,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 947, in _gradcheck_real_imag
    gradcheck_fn(func, func_out, tupled_inputs, outputs, eps,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1168, in _fast_gradcheck
    _check_analytical_numerical_equal(analytical_vJu, numerical_vJu, complex_indices,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1148, in _check_analytical_numerical_equal
    raise GradcheckError(_get_notallclose_msg(a, n, j, i, complex_indices, test_imag, is_forward_ad) + jacobians_str)
torch.autograd.gradcheck.GradcheckError: Jacobian mismatch for output 0 with respect to input 1,
numerical:tensor(0.7871)
analytical:tensor(0.2123)

The above quantities relating the numerical and analytical jacobians are computed
in fast mode. See: https://github.com/pytorch/pytorch/issues/53876 for more background
about fast mode. Below, we recompute numerical and analytical jacobians in slow mode:

Numerical:
 tensor([[1.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 1.0000, 1.0000,  ..., 0.0000, 0.0000, 0.0000],
        ...,
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 1.0000, 1.0000]])
Analytical:
tensor([[1., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        ...,
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 1.]])

The max per-element difference (slow mode) is: 1.0000000000000009.


======================================================================
ERROR: test_fn_grad_nn_functional_max_unpool2d_cuda_float64 (__main__.TestGradientsCUDA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_utils.py", line 1422, in wrapper
    method(*args, **kwargs)
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_utils.py", line 1422, in wrapper
    method(*args, **kwargs)
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
    raise rte
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 371, in instantiated_test
    result = test(self, **param_kwargs)
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 737, in test_wrapper
    return test(*args, **kwargs)
  File "/home/krshrimali/pytorch/test/test_ops.py", line 603, in test_fn_grad
    self._grad_test_helper(device, dtype, op, op.get_op())
  File "/home/krshrimali/pytorch/test/test_ops.py", line 586, in _grad_test_helper
    return self._check_helper(device, dtype, op, variant, 'gradcheck', check_forward_ad=check_forward_ad,
  File "/home/krshrimali/pytorch/test/test_ops.py", line 559, in _check_helper
    self.assertTrue(gradcheck(fn, gradcheck_args,
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_utils.py", line 2718, in gradcheck
    return torch.autograd.gradcheck(fn, inputs, **kwargs)
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1273, in gradcheck
    return _gradcheck_helper(**args)
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1286, in _gradcheck_helper
    _gradcheck_real_imag(gradcheck_fn, func, func_out, tupled_inputs, outputs, eps,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 947, in _gradcheck_real_imag
    gradcheck_fn(func, func_out, tupled_inputs, outputs, eps,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1168, in _fast_gradcheck
    _check_analytical_numerical_equal(analytical_vJu, numerical_vJu, complex_indices,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1148, in _check_analytical_numerical_equal
    raise GradcheckError(_get_notallclose_msg(a, n, j, i, complex_indices, test_imag, is_forward_ad) + jacobians_str)
torch.autograd.gradcheck.GradcheckError: Jacobian mismatch for output 0 with respect to input 0,
numerical:tensor(0.3207, device='cuda:0')
analytical:tensor(0.9777, device='cuda:0')

The above quantities relating the numerical and analytical jacobians are computed
in fast mode. See: https://github.com/pytorch/pytorch/issues/53876 for more background
about fast mode. Below, we recompute numerical and analytical jacobians in slow mode:

Numerical:
 tensor([[1.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 1.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        ...,
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 1.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],
       device='cuda:0')
Analytical:
tensor([[1., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 1.,  ..., 0., 0., 0.],
        [0., 0., 1.,  ..., 0., 0., 0.],
        ...,
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 1., 0.],
        [0., 0., 0.,  ..., 0., 1., 0.]], device='cuda:0')

The max per-element difference (slow mode) is: 1.0.


======================================================================
ERROR: test_fn_gradgrad_nn_functional_max_unpool2d_cuda_float64 (__main__.TestGradientsCUDA)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_utils.py", line 1422, in wrapper
    method(*args, **kwargs)
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_utils.py", line 1422, in wrapper
    method(*args, **kwargs)
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 376, in instantiated_test
    raise rte
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 371, in instantiated_test
    result = test(self, **param_kwargs)
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_device_type.py", line 737, in test_wrapper
    return test(*args, **kwargs)
  File "/home/krshrimali/pytorch/test/test_ops.py", line 625, in test_fn_gradgrad
    self._gradgrad_test_helper(device, dtype, op, op.get_op())
  File "/home/krshrimali/pytorch/test/test_ops.py", line 591, in _gradgrad_test_helper
    return self._check_helper(device, dtype, op, variant, 'gradgradcheck')
  File "/home/krshrimali/pytorch/test/test_ops.py", line 569, in _check_helper
    self.assertTrue(gradgradcheck(fn, gradcheck_args,
  File "/home/krshrimali/pytorch/torch/testing/_internal/common_utils.py", line 2738, in gradgradcheck
    return torch.autograd.gradgradcheck(fn, inputs, grad_outputs, **kwargs)
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1402, in gradgradcheck
    return gradcheck(
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1273, in gradcheck
    return _gradcheck_helper(**args)
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1286, in _gradcheck_helper
    _gradcheck_real_imag(gradcheck_fn, func, func_out, tupled_inputs, outputs, eps,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 947, in _gradcheck_real_imag
    gradcheck_fn(func, func_out, tupled_inputs, outputs, eps,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1168, in _fast_gradcheck
    _check_analytical_numerical_equal(analytical_vJu, numerical_vJu, complex_indices,
  File "/home/krshrimali/pytorch/torch/autograd/gradcheck.py", line 1148, in _check_analytical_numerical_equal
    raise GradcheckError(_get_notallclose_msg(a, n, j, i, complex_indices, test_imag, is_forward_ad) + jacobians_str)
torch.autograd.gradcheck.GradcheckError: Jacobian mismatch for output 0 with respect to input 1,
numerical:tensor(1.0067, device='cuda:0')
analytical:tensor(0.3634, device='cuda:0')

The above quantities relating the numerical and analytical jacobians are computed
in fast mode. See: https://github.com/pytorch/pytorch/issues/53876 for more background
about fast mode. Below, we recompute numerical and analytical jacobians in slow mode:

Numerical:
 tensor([[1.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 1.0000, 1.0000,  ..., 0.0000, 0.0000, 0.0000],
        ...,
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 1.0000, 1.0000],
        [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],
       device='cuda:0')
Analytical:
tensor([[1., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 1., 0.,  ..., 0., 0., 0.],
        ...,
        [0., 0., 0.,  ..., 0., 0., 0.],
        [0., 0., 0.,  ..., 0., 1., 0.],
        [0., 0., 0.,  ..., 0., 0., 0.]], device='cuda:0')

The max per-element difference (slow mode) is: 1.0000000000000009.

To Reproduce

Steps to reproduce the behavior:

1. Checkout to the branch: https://github.com/krshrimali/pytorch/tree/origin/max_unpoolNd
2. Run the OpInfo tests after removing the skips for max_unpool2d. python3 test/test_ops.py -k max_unpool2d

Expected behavior

The tests should pass.

Additional context

Please see #67328 and #67328 for the discussion.

cc: @albanD @nikitaved

cc @ezyang @gchanan @zou3519 @bdhirsh @jbschlosser @albanD @gqchen @pearu @nikitaved @soulitzer @lezcano @Varal7 @mruberry @walterddr

[mruberry]

Thanks for reporting this issue, @krshrimali. Tentatively marking "high priority" as the error seems large enough to possibly be a silent correctness issue. @albanD what do you think?

[nikitaved]

@mruberry , we had a short discussion about this issue here: #67328 (comment).
I could have a look. It also seems to me we can avoid custom kernels and use already implemented advanced indexing operators. I speculate it will be as fast and hopefully correct :)