Wrap padding

[gcskoenig]

Hi all,

I could not find any padding functionality that equals the "wrap" padding from numpy.

E.g. this:
[[0 1 2]
[3 4 5]
[6 7 8]]

Shall be padded like this:
[[8 6 7 8 6]
[2 0 1 2 0]
[5 3 4 5 3]
[8 6 7 8 6]
[2 0 1 2 0]]

Would be of great value!

[zou3519]

I think this is what you want:

import torch
n = 3  # n by n matrix
x = torch.Tensor([[0, 1, 2],[3, 4, 5],[6, 7, 8]])
x.repeat(n, n)[(n-1):(2*n+1), (n-1):(2*n+1)]

Idea: repeat the tensor in a 3x3 grid, then crop out the part you need.

[harpone]

Here's a simple and pretty fast implementation for [H, W] shaped arrays:

def pad_circular(x, pad):
    """

    :param x: shape [H, W]
    :param pad: int >= 0
    :return:
    """
    x = torch.cat([x, x[0:pad]], dim=0)
    x = torch.cat([x, x[:, 0:pad]], dim=1)
    x = torch.cat([x[-2 * pad:-pad], x], dim=0)
    x = torch.cat([x[:, -2 * pad:-pad], x], dim=1)

    return x

For a 1000x1000 array I get this:

N = 1000

x = torch.randn(N, N).cuda()

%timeit pad_circular(x, 1)

> 38 µs ± 1.53 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

For the repeat way above I get CUDA OOM error...

[jiaqi-jiang]

Here's a simple and pretty fast implementation for [H, W] shaped arrays:

def pad_circular(x, pad):
    """

    :param x: shape [H, W]
    :param pad: int >= 0
    :return:
    """
    x = torch.cat([x, x[0:pad]], dim=0)
    x = torch.cat([x, x[:, 0:pad]], dim=1)
    x = torch.cat([x[-2 * pad:-pad], x], dim=0)
    x = torch.cat([x[:, -2 * pad:-pad], x], dim=1)

    return x

For a 1000x1000 array I get this:

N = 1000

x = torch.randn(N, N).cuda()

%timeit pad_circular(x, 1)

> 38 µs ± 1.53 µs per loop (mean ± std. dev. of 7 runs, 10000 loops each)

For the repeat way above I get CUDA OOM error...

Is it autograd compatible? Do we need to implement backward function for circular padding?

[harpone]

@Douphoton it's just slicing and concatenation so it's differentiable.

BTW I made a mistake in the repeat implementation, now I got 263 µs ± 853 ns, so the "manual" version above is not that much faster...
EDIT: but of course this version also uses much less memory