About the consistency loss

[yakexee]

Hi, thanks for sharing your code. For the function F.kl_div(), the first parameter is input and the second is target. I am confused why the target is not p_mixture on L401-L403?
Thanks.

[wildphoton]

Hi @yakexee, I think F.kl_div() defines inputs and targets in the way how NLL loss is used. Since NLL(input, target) = cross_entropy(target, input). I believe F.kl_div(input, target) = KL(target || input). You can check this with an easy example by comparing the results with a KL div function implemented by yourself. Since we are computing KL(p_aug || p_mixture), we called F.kl_div(p_mixture, p_aug). FYI, this loss is adapted from the Augmix's implementation. I hope it helps

[yakexee]

Thanks for the kind reply. That helps!