FreeMatch SAF loss is a negative value?

[hzhz2020]

Bug

In your FreeMatch paper equation 11, the SAF loss is a negative value. Then the SAF loss is added to the total loss. Isnt this causing SumNorm (p˜t/h˜t) and SumNorm(p¯/h¯) to be more dis-simmilar?
�

Reproduce the Bug

Error Messages and Logs

[Hhhhhhao]

Negative loss doesn't affect the gradient

[hzhz2020]

thank you for your fast reply!
I am just curious about the intuition behind this loss. By minimizing $L_{f}$ in eq11, aren't you pushing SumNorm (p˜t/h˜t) and SumNorm(p¯/h¯) to be further apart? I thought you want to make them closer?

[Hhhhhhao]

thank you for your fast reply! I am just curious about the intuition behind this loss. By minimizing Lf in eq11, aren't you pushing SumNorm (p˜t/h˜t) and SumNorm(p¯/h¯) to be further apart? I thought you want to make them closer?

The loss encourages fairness on average predictions. We expect the average predictions to be close to uniform. Reflected on the entropy loss, it corresponds to maximizing the entropy. We replace the target term with a momentum-smoothed average prediction for stability. That's the intuition behind this loss. I think the paper and relevant works mentioned in the paper discussed this in more detail.

[hzhz2020]

thank you for explaining it!