Support new target type Byte in NLLLoss

[wangkuiyi]

Only a LongTensor may be used as the target parameter of nn.NLLLoss.forward. Runtime errors would occur while using Char/Byte/ShortTensor, as shown by the following code snippet.

e = torch.Tensor([[math.log(3/4), math.log(1/4)]])
t = torch.LongTensor([0])
torch.nn.NLLLoss()(e, t)

t = torch.CharTensor([0])
torch.nn.NLLLoss()(e, t) # RuntimeError: expected scalar type Long but found Char

t = torch.ByteTensor([0])
torch.nn.NLLLoss()(e, t) # RuntimeError: expected scalar type Long but found Byte

I really want to store pre-computed targets as byte tensors to save memory. Is it possible to have NLLLoss accept additional types of targets?

I am requesting byte-typed target because adding a type would enlarge the binary by one C++ template initialization, and we could divide a wide softmax into a hierarchy of much shorter (say 256) ones.

cc @ezyang @gchanan @zou3519 @bdhirsh @jbschlosser @anjali411 @Varal7 @ngimel @albanD @mruberry

[mruberry]

cc @jbschlosser @thomasjpfan

Ideally this should be ported to ATen if that's not too much extra work, otherwise adding this feature in THCNN is OK

[ezyang]

FB only: this issue tracks the internal request at https://fb.workplace.com/groups/1405155842844877/permalink/4785771701449924

My personal opinion: if we don't want to add this because of CUDA binary size, we should make it work with JITerator.

[jbschlosser]

@thomasjpfan is prioritizing the support for byte targets (ideally alongside migration to ATen). We are okay adding just one more target type (one more template instantiation), but beyond that, this should be solved with JITerator as @ezyang mentioned.