Index / SetValue don't work for 1D Variables

[fmassa]

Snippet to reproduce.

import torch
from torch.autograd import Variable

x = Variable(torch.Tensor([1, 2, 3]), requires_grad=True)
y = x[0]

y.backward() # error

x[1] = 5 # error

The problem is in here, here and here, and is due to the fact that torch.Tensor return numbers when indexing a 1D tensor (which I think is the desired behaviour as we don't have broadcasting yet?).

Maybe we should check if the input is 1D (and that the index is of range 1) and to use narrow instead of select in this case?
Autograd Variables return a 1D tensor anyway if we index it with a 1D tensor in forward, but this solution could avoid the sync point with cuda tensors in Index (as we don't return a number).

Or, another solution would be to have something like 0D tensors which behave almost like numbers? This seems to be what numpy does.

[fmassa]

One thing to have in mind, if we only add the narrow thing I mentioned, some more special treatment will need to be added if we want to do something like

import torch
from torch.autograd import Variable

x = Variable(torch.Tensor([1, 2, 3]), requires_grad=True)
y = Variable(torch.Tensor([2, 3, 4]), requires_grad=True)
x[0] = y[1] # this would error out with narrows
x[y[0]] = 1 # error as well, but less common

The issue first issue would be that we pass a 1D tensor as the filling value to the fill function (instead of a number).
The second one would be that we would be passing a 1D tensor as argument of the index (or narrow) function instead of a number as well.

Getting the number back from the tensor would be a sync point on the GPU (as is already the case btw), and might make the code pretty ugly with lots of checks here and there asserting for 1d tensors that are supposed to be numbers.

I think this is a corner case though, as maybe not many people will be changing single elements of the tensors (would be slow!).

[apaszke]

Resolution of this issue will depend on what we decide about #144.

[apaszke]

Fixed in #176.