loss of paddding

[liujiqiang999]

Hi, do we need to ignore padding's loss when we do back-translation? It seems that the code doesn't ignore the padding when we calculate loss. Thank you very much.

XLM/src/trainer.py

Line 824 in 14fe2d4

pred_mask = alen[:, None] < len1[None] - 1 # do not predict anything given the last target word

[glample]

We do no? This line is supposed to set to 0 everything that is longer than sentences in x1, i.e. padding.

[liujiqiang999]

Ah, right. I got. Thanks.