tensor.new() can have nan's, and some pytorch code is thus unsafe

[ankitkv]

tensor.new() does not initialize memory, so it could end up containing nan's. This could be unsafe in some cases.

For example, in torch/autograd/variable.py:

def mm(self, matrix):
    output = Variable(self.data.new(self.data.size(0), matrix.data.size(1)))
    return self._static_blas(Addmm, (output, 0, 1, self, matrix), False)

def bmm(self, batch):
    output = Variable(self.data.new(self.data.size(0), self.data.size(1),
                                    batch.data.size(2)))
    return self._static_blas(Baddbmm, (output, 0, 1, self, batch), False)

def mv(self, vector):
    output = Variable(self.data.new(self.data.size(0)))
    return self._static_blas(Addmv, (output, 0, 1, self, vector), False)

def ger(self, vector):
    output = Variable(self.data.new(self.data.size(0), vector.data.size(0)))
    return self._static_blas(Addr, (output, 0, 1, self, vector), False)

is dangerous because output could contain a nan, and even though alpha is being set to 0, nan * 0 = nan and the result could contain a nan (I had an optim test failing because of a nan originating this way).

I haven't done an exhaustive search, so there may be other places in the code that could have this issue.

[soumith]

inside the blas side of things, we explicitly check for the special case of alpha=0 and zero the output.

if you see that it isn't true for a particular blas call, let me know and I'll fix it, but for mm and mv it should already be fixed.

[ankitkv]

I see. I had the issue with ger, can you check?

[ankitkv]

For instance, undoing the zero'ing of the new tensors in the four functions mentioned in the opening post causes tests to fail in #1306. The tests pass on one of the python versions, and on my machine a different test suit fails (which is where I hunted down the root). Now, it seems very possible that I introduced this vulnerability somehow, because this doesn't seem to happen without my PR. I'll look into it.

[ngimel]

For ger, if beta=0 tensors are explicitly zeroed on the CPU and GPU sides:
https://github.com/pytorch/pytorch/blob/master/torch/lib/THC/generic/THCTensorMathBlas.cu#L158
https://github.com/pytorch/pytorch/blob/master/torch/lib/TH/generic/THTensorMath.c#L1369

[ankitkv]

Ok, I confirmed that this is not a problem without my PR (I manually inserted nan's into output to test).