Grads needed

[alexbw]

• repeatTensor
• max
• min
• cat
• new
• fill
• index
• __index (operator b = a[...])

[clementfarabet]

note: torch.cat implemented but only with 2 inputs (table not supported – table support was added to the public branch recently)

[alexbw]

Shouldn't need fill new and copy, since those produce tensors that, if used in computation, will be wrapped in nodes. If not used, and sent as output, then output will be independent of input.

[nkoumchatzky]

What do you mean by newindex here?

[clementfarabet]

Just clarified on the index/newindex.

[nkoumchatzky]

Got it.

[clementfarabet]

@alexbw got it, I removed indexCopy. Ok so let me try a few of these things then, last time I tried to do a copy in a narrowed tensor it failed.

[clementfarabet]

No, confirming that copy doesn't work as is. Pushing a new test to capture it: 0db08d8

[clementfarabet]

So adding back copy and indexCopy. These would work if you copied a tensor into them for which you didn't need grads... but in if you do, because you copy something that depends on a variable for which you want grads, then it needs to be defined.

Also I started a branch copy a few days ago to implement the copy operator, but I couldn't get it to work. What am i missing there?

[clementfarabet]

And once copy works, then __newindex__ is just syntactic sugar for :narrow():copy() and :select():copy().

[alexbw]

So you mean copy needs to be overridden to carry grads with the tensor?

[clementfarabet]

Yes it does! I pushed the code I have in master, which defines the 2 grads for torch.copy; it's failing with a A node type was not returned. This is either because a gradient was not defined, or the input is independent of the output.

[clementfarabet]

indexCopy will also need that, but you're right that indexFill won't. indexFill is just like any other constant source.

[alexbw]

Not really a very focused issue, closing.