About the coefficient 1000.0 in DKD.py

[ywher]

in line 21 - 26 in DKD.py

pred_teacher_part2 = F.softmax(
    logits_teacher / temperature - 1000.0 * gt_mask, dim=1
)
log_pred_student_part2 = F.log_softmax(
    logits_student / temperature - 1000.0 * gt_mask, dim=1
)

What dose the coefficient 1000.0 represent? Is it the number of classes ?

[cailinhang]

What dose the coefficient 1000.0 represent? Is it the number of classes ?

I guess that softmax(-1000) ≈ 0. Thus -1000 is just used to make the probability of the target class to be around 0.

[Zzzzz1]

It's an implementation for masking out the target logits (only keeping the non-target logits). A more readable way is "pred = F.softmax(logits[1-gt_mask] / temperature)", yet we find it would cost a lot of time because of the indexing operation.