repeat_interleave throws IndexError on repeating size-zero dimension

[jhultman]

🐛 Bug

torch.repeat_interleave(input, repeats, dim) throws IndexError when input.size(dim) == 0. The corresponding numpy.repeat handles the size-zero case gracefully.

To Reproduce

import torch
x = torch.zeros((5, 0))
y = torch.repeat_interleave(x, repeats=3, dim=1)
assert (y.shape == (5, 0))
>> IndexError: select(): index -1 out of range for tensor of size [0] at dimension 0

Expected behavior

import numpy
x = numpy.zeros((5, 0))
y = numpy.repeat(x, repeats=3, axis=1)
assert (y.shape == (5, 0))

Environment

PyTorch version: 1.1.0
CUDA used to build PyTorch: 9.0.176
OS: Ubuntu 16.04.4 LTS
Python version: 3.7
cuDNN version: /usr/lib/x86_64-linux-gnu/libcudnn.so.7.1.4

(I have also tried on torch nightly version 1.2.0.dev20190711 and it has not been fixed.)

Additional context

I think it would be more consistent and intuitive if output.size(dim) == 0 whenever input.size(dim) * repeats == 0. The output size should be determined by the product of the input size in the appropriate dimension and the repeats (broadcasted). This is how the numpy version is designed.

For context, the converse works fine since the zero-dim does not participate:

import torch
x = torch.zeros((5, 0))
y = torch.repeat_interleave(x, repeats=3, dim=0)
assert (y.shape == (15, 0))

This one throws IndexError but is supported in numpy:

import torch
x = torch.tensor([], dtype=torch.int64)
y = torch.repeat_interleave(x, x)
>> IndexError: select(): index -1 out of range for tensor of size [0] at dimension 0

I think it makes sense for "repeating empty by empty" to vacuously yield empty (of same size as input).

[jhultman]

Sorry if off-topic but regarding unit tests:

Speaking for myself, one of the nice features of the torch 1.0 release was expanded support for operations involving 0-element n-dimensional tensors (see #9947). But has there been a review of test_torch.py to ensure all applicable functions have test cases with 0-element tensors? Maybe a few have slipped through the cracks.

For example I checked test_repeat_interleave in test_torch.py and don't see any test cases with input.numel() == 0. For comparison, the related function torch.expand does have one test for "expanding empty to empty", but nothing for "empty to non-empty", or "non-empty to empty".

[gchanan]

@jhultman yes we did add tests for all the operators we decided to add support for as part of #9947 and related PRs. That doesn't cover every single operator (long tail problem...) and the specific operator in question was added after the general feature (so should have included tests as part of its PR).

But having general operator-sized testing would help a lot here.

[jhultman]

@gchanan General operator-sized testing sounds like a very nice idea. What would that look like? I can open a new issue if you think appropriate.

[gchanan]

My rough thought is we should have a DSL or something that describes allowed input sizes and expected output sizes. That would also be useful for lazy tensors. And then we could drive automated, randomized testing based on that.