You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The accuracy metric reported by the Evaluator for masked LM task during LM finetuning is wrong. This is due to the fact that we have there a nested list of predictions with different lengths (different number of masked tokens per sample). This leads to wrongly converted numpy arrays and a wrong metric. I am working on a fix.
The text was updated successfully, but these errors were encountered:
The accuracy metric reported by the Evaluator for masked LM task during LM finetuning is wrong. This is due to the fact that we have there a nested list of predictions with different lengths (different number of masked tokens per sample). This leads to wrongly converted numpy arrays and a wrong metric. I am working on a fix.
The text was updated successfully, but these errors were encountered: