Attention Weight Bug in Embedding

[kandorm]

https://github.com/xhuang31/KEQA_WSDM19/blob/f923a15cc732e8844c26e4c653c306c90e067734/embedding.py#L56

There is a bug in this line, because 'self.attn(torch.cat((x, outputs), 1)' generate a tensor which size is
(seq_len*batch_size, 1), which results in all softmax results to be 1.0.

[xhuang31]

Yes. You are right. It should be

attn_weights = F.softmax(self.attn(torch.cat((x, outputs), 1)), dim=0).

Then we would need different hyperparameters. I will leave this issue open and fine tune the updated model when I get time.

Thanks for your comments.