Why is a custom label smoothing used instead of CrossEntropy from torch?

[janvainer]

Torch CrossEntropy allows using label smoothing: https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html

Is there any difference between the icefall implementation and the one provided by pytorch?

[csukuangfj]

The argument label_smoothing for torch.nn.CrossEntropyLoss is only available for torch >= 1.10

It is added in https://github.com/pytorch/pytorch/pull/63122/files#diff-6b8e21a44da970b14c1d62eb1747af1b09d3914ff1f6354754c62f1f5784cb9dR1107
on 2021-08-12, which is quite new.

[janvainer]

I see, but otherwise the functionality should be the same, right?

[csukuangfj]

I have not checked that. Will read PyTorch's implementation tomorrow and come back later.

Thanks for pointing this out. I was not aware of this argument for torch.nn.CrossEntropyLoss.

[janvainer]

Thanks, yeah this torch version is quite new. I found it by accident :)

[csukuangfj]

Is there any difference between the icefall implementation and the one provided by pytorch?

Please see #107