The issue for the loss of regression tasks

[ChuanMeng]

LM-BFF/src/models.py

Line 184 in c282f52

loss_fct = nn.KLDivLoss(log_target=True)

Hi Tianyu,
Thank you for releasing the code.
I found one problem with the loss of regression tasks.

"logits" is through the operation "logsoftmax" and is in [-infinite, 0], while "labels" is not through that operation and is always greater than 0. They are not in the same space, so I think here log_target cannot be True. Or the "labels" should be operated by "torch.log(labels+very small number)".

What do you think of it?

Look forward to your reply.

Best wishes,
Chuan

[gaotianyu1350]

Hi Chuan,

Thanks for your interest in our work! It seems that you are right, and I have no idea why the wrong version actually worked! Please let me know if you try the other way and it works better!