Student should set to eval() before calculating s_logits_l ?

[Vurkty]

If you don't do this, model's output will be random because of the dropout etc.
Then your s_loss_l_new and s_loss_l_old will be random too and your dot_production won't give right feedback.
Maybe this issue affects your results

[kekmodel]

I'll test this. Thank you!

[kekmodel]

Test accuracy is 94.0.

[Vurkty]

Well, it seems not work.
Thanks for your test!