conv1d and conv2d should support further batching

[dsyme]

It looks like conv1d and conv2d expect only one dimension of batching (NxCxI to conv1d). In torch you can do additional dimensions of batching (N1xN2xCxI to conv1d), compare:

import torch
import torch.nn.functional as F
x = torch.zeros(3,1,4,4, dtype=torch.float32)
y = torch.zeros(3,1,4,4, dtype=torch.float64);
F.conv1d(x,y)

and

let x = dsharp.zeros([3;1;4;4])
let y = dsharp.zeros([3;1;4;4])
dsharp.conv1d(x,y)

gives

System.Exception: Expecting two 3d Tensors t1, t2 where t1 is input (NxCxI: batchSize x inputChannels x inputLength) and t2 is filters (KxCxF: outputChannels x inputChannels x kernelLength), received Tensors with shapes [|3; 1; 4; 4|], [|3; 1; 4; 4|]

[gbaydin]

I think the DiffSharp implementations are consistent with
https://pytorch.org/docs/stable/nn.functional.html#conv1d

https://pytorch.org/docs/stable/nn.functional.html#conv2d

except the "groups" feature. We can revisit this.

[gbaydin]

I'm currently working on 3d convolutions and then plan to move on to other essential operations in standard ML pipelines. I would like to finish that work before addressing this.

[dsyme]

right no rush with this

[moloneymb]

Supporting 'further batching' should be a byproduct of implementing a reshape operation. The reshape operation should not add significant overhead as it just changes the shape.

[gbaydin]

I investigated this further and I believe that this is not needed and we can close this issue.

Details:

This is how a regular 1d conv looks like in pytorch:

import torch
x = torch.zeros(3,2,16, dtype=torch.float32) # input minibatch:3, in_channels: 2, size: 16
y = torch.zeros(5,2,3, dtype=torch.float32) # kernel out_channels: 5, in_channels: 2, size: 3
torch.nn.functional.conv1d(x,y).shape

The result is

torch.Size([3, 5, 14]) # minibatch: 3, out_channels: 5, size: 14

As @dsyme observed one could also use more than 3d tensors with this op, for example

x = torch.zeros(3,2,10,16, dtype=torch.float32) # input minibatch:3, in_channels: 2, ?: 10, size: 16
y = torch.zeros(5,2,10,3, dtype=torch.float32) # kernel out_channels: 5, in_channels: 2, ?: 10, size: 3
torch.nn.functional.conv1d(x,y).shape

which gives

torch.Size([3, 5, 1, 14]) # minibatch: 3, out_channels: 5, ?: 1, size: 14

or another example

x = torch.zeros(3,2,10,20,16, dtype=torch.float32) # input minibatch:3, in_channels: 2, ?: 10, ?: 20, size: 16
y = torch.zeros(5,2,10,20,3, dtype=torch.float32) # kernel out_channels: 5, in_channels: 2, ?: 10, ?: 20, size: 3
torch.nn.functional.conv1d(x,y).shape

which gives

torch.Size([3, 5, 1, 1, 14]) # minibatch: 3, out_channels: 5, ?: 1, ?: 1, size: 14

The examples suggests that the first and second dimensions are still used as the minibatch and channel dimensions (not extra batch dimensions), the last dimension is still used as the data (e.g., image) dimension and any dimensions in between are somehow ignored in the output.

This behavior is undocumented here and quite strange. I think we should not implement this and close the issue. I would support implementing it if it was documented in pytorch docs with an explanation of what it actually does.

[gbaydin]

Closing this. Please ping me if there is something I missed and we can reopen.