gradgradcheck for torch.repeat and torch.tile is outrageously slow

[mruberry]

torch.repeat and torch.tile (which is implemented using torch.repeat) are relatively fast compared to NumPy's torch.tile, but attempting to gradgradcheck them is incredible slow in some cases. For example:

x = torch.randn(5, 5, 5, requires_grad=True, dtype=torch.double)

def partial(x):
    return x.repeat(5, 5, 5, 5)

gradgradcheck(partial, x)

takes 77.93s on my devfair! While not an apples to apples comparison, computing the function's Hessian is relatively fast:

x = torch.randn(5, 5, 5, requires_grad=True, dtype=torch.double)

def partial(x):
    return x.repeat(5, 5, 5, 5).sum()

torch.autograd.functional.hessian(partial, x)

takes only .07s to run. That is, it is 1000x faster than the gradgradcheck.

gradgradcheck being so slow appears to have a real impact. See those two tests on ASAN:

Dec 30 02:18:32   test_tile_more_reps_dims_cpu (__main__.TestAutogradDeviceTypeCPU) ... ok (1023.734s)
Dec 30 02:20:21   test_tile_same_reps_dims_cpu (__main__.TestAutogradDeviceTypeCPU) ... ok (109.164s)

or on the clang build:

Dec 30 01:51:11   test_tile_more_reps_dims_cpu (__main__.TestAutogradDeviceTypeCPU) ... ok (115.829s)
Dec 30 01:51:24   test_tile_same_reps_dims_cpu (__main__.TestAutogradDeviceTypeCPU) ... ok (13.091s)

cc @ezyang @albanD @zou3519 @gqchen @pearu @nikitaved @soulitzer @mruberry @VitalyFedyunin @walterddr

[albanD]

Well, the .sum() makes all the difference.
You can reduce the size of the input/output to reduce this.

[anjali411]

reduced the input size for tile tests and they don't timeout anymore!