R2Conv breaks with kernel_size=2 (list index out of range)

[mpaepper]

This looks like a really nice library and I want to test it out, so I am converting an existing neural network to a rotational invariant one.

In one of my layers, I have a convolution with kernel_size=2. This breaks in the R2Conv while kernel_size=3 or more works fine.

Is there a reason why a kernel_size=2 convolution cannot work in the rotational setting or is this a bug?

Minimal example:

    r2_act = gspaces.Rot2dOnR2(N=8)
    in_type = nn.FieldType(r2_act, 3*[r2_act.trivial_repr])
    out_type = nn.FieldType(r2_act, 6*[r2_act.regular_repr])

    conv = nn.R2Conv(in_type, out_type, kernel_size=2, initialize=False, padding=0).cuda()
    x = torch.ones(size=[1,3,190,190]).cuda()
    x = nn.GeometricTensor(x, in_type)
    conv(x)

Error:

list index out of range in basisexpansion_blocks.py in line 327

[Gabri95]

Hi @mpaepper,

Thanks!

While 2x2 convolutional filters are supported, they are not really stable to continuous rotations.
The problem you experienced comes from the fact that the default method which constructs the basis for the filters considers the 2x2 grid too small to sample any filter.
As a result, the basis is empty.

I tuned manually this method a long time ago and it works pretty well for odd size filters and with larger even size filters.
The 2x2 grid seems a corner case, but I would not recommend using this grid if your goal is achieving equivariance to all rotations.

I definetly need to handle this kind of error better. I will add more informative error messages and I will raise it at initialization of the model.
Thanks for reporting this!

If you have to use 2x2 filters, here are a few more details about what is going on and a few suggestions.

The 2x2 filter can be useful if you only care about equivariance to 90 degrees rotations (i.e. N=4).
In that case, you can play with the sigma, rings and frequencies_cutoff parameters of R2Conv.
Maybe, by tuning these parameters a bit, you might even be able to get something which is stable enough to other rotations for your application (but I didn't experiment here enough).

A steerable filter is in general split in multiple rings, see Figure 2 in https://arxiv.org/pdf/1711.07289.pdf
The parameter rings is a list of float, defining the radius of the different rings the filter is split into.
The parametr sigma defines the width of each of them. You can pass a list containing a different sigma per radius or just a singular float value which is used for all rings.
The parameter frequencies_cutoff regulates the maximum frequency on each ring. By default, it uses the policy I manually tuned. If you pass a float value frequencies_cutoff=F, the maximum frequency at radius r becomes int(r * F).

A 2x2 grid is represented as 4 points at coordinates (-1/2, -1/2), (1/2, -1/2), (1/2, 1/2) and (-1/2, 1/2), where the rings are evaluated.
In this case, by default, my method chooses rings=[0.0] and sigma=0.005 (this low value for sigma is usually used only for radius=0.0 and is useful for odd size filters, but it is problematic for 2x2 filters).
This results in a basis whose filters evaluate to something too close to 0.0 on the 4 points of the grid, so they are all discarded.

You can fix this by either choosing a larger value of sigma (e.g. 0.2).
This still gives you filters with maximum frequency 0 (i.e. isotropic filters) because they are centered at radius=0.
This might be the most stable solution if you are still interested in rotations smaller than 90 degrees, but isotropic filters are not really expressive...

 conv = nn.R2Conv(in_type, out_type, kernel_size=2, initialize=False, padding=0, sigma=0.2)

As long as sigma>=0.2, the basis will contain just the contant filter (same value on all 4 points of the grid).
Higher value of sigma won't change the basis because it's anyways renormalized later, so no need to tune sigma in this case.

Otherwise, you can also choose rings=[0.7] (in this case, sigma=0.4 by default) and set frequencies_cutoff=1.5.
This allows you to also have frequency 0 and 1 filters in your basis.

 conv = nn.R2Conv(in_type, out_type, kernel_size=2, initialize=False, padding=0, rings=[0.7], frequencies_cutoff=1.5)

Hope this helps!

Let me know if you have more questions

Gabriele

[mpaepper]

Thank you Gabriele,

I really appreciate the detailed explanation and have a better understanding now, so I can make some experiments.

[Gabri95]

Nice, I am happy this was helpful!
I am closing this issue then.
Please, open another one if you find others problems or you need help :)