Difference between dropout2d and dropout3d

[kianasun]

🐛 Bug

The difference between Dropout2d and Dropout3d is not clear. They use exactly same code but different documentation. As in the documentation of Dropout2d and Dropout3d, they should have different shapes of input, but it seems that they can work for inputs with other shape (dimension > 2). Should we just combine them (a generalized one)? Or at least restrict the input?

To Reproduce

torch.manual_seed(0)
m = nn.Dropout3d(p=0.2)
input = torch.randn(2, 2, 2)
output = m(input)
output
tensor([[[ 1.9262, -0.3668],
         [-2.7235,  0.7105]],

        [[-1.3557, -1.7482],
         [ 0.5042,  1.0475]]])

torch.manual_seed(0)
m = nn.Dropout2d(p=0.2)
input = torch.randn(2, 2, 2)
output = m(input)
output
tensor([[[ 1.9262, -0.3668],
         [-2.7235,  0.7105]],

        [[-1.3557, -1.7482],
         [ 0.5042,  1.0475]]])

[zou3519]

We should definitely at leastrestrict the input. Under the hood, Dropout2d and Dropout3d call the same function feature_dropout that interprets the input tensor as having batch dimension 0 and channel dimension 1. This means that sending a 5d batch of images (N0, N1, C, H, W) to Dropout2d is misleadingly incorrect.

[kianasun]

@zou3519 Yes. If we simply want to restrict the input, maybe I can add some TORCH_CHECK. If we want to generalize it, we can just give some examples to show how it works in different situations, e.g. images.