You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Could someone help me understand why is the BERTScore recall is non zero when there is an empty candidate sentence. Isn't zero the right score for precision as well as recall over here?
score([''], ['hello how are you'], lang='en')
This is what the model returns now:
Warning: Empty candidate sentence detected; setting precision to be 0. (tensor([0.]), tensor([0.7738]), tensor([0.]))
Good catch! Thanks a lot! It comes from the fact that the recall uses the [CLS] and [EOS] tokens added to the empty sentences, so the scores are not zero. However, I believe it is more reasonable to set it as zero. I have fixed this bug in a new commit b1a346.
Hi,
Could someone help me understand why is the BERTScore recall is non zero when there is an empty candidate sentence. Isn't zero the right score for precision as well as recall over here?
score([''], ['hello how are you'], lang='en')
This is what the model returns now:
Warning: Empty candidate sentence detected; setting precision to be 0.
(tensor([0.]), tensor([0.7738]), tensor([0.]))
best,
Anupama
Originally posted by @achingacham in #62 (comment)
The text was updated successfully, but these errors were encountered: