You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We get incorrect evaluation using M2Scorer as well (please double check).
Hypothesis:
It 's Thursday , is n't it ?
M2 file:
S Thursday , is it not ?
A 0 0|||Mec|||It 's|||REQUIRED|||-NONE-|||0
A 3 5|||Mec|||n't it|||REQUIRED|||-NONE-|||0
Based on my understanding, the reason is the initialization step of the levenshtein matrix. It creates the first row using the index of the hypothesis, i.e: the necessary edits to transform an empty sentence into the hypothesis. I think the start and end of such edits should always be 0.
My proposed solution is to change line 827of file levenshtein.py into:
edit = ("ins", 0, 0, '', second[j-1], 0) # always insert at the beginning
It appears to me that the modification will fix the problem, yet I am not sure whether other errors may arise.
For example:
S Thursday , is it not ?
A 0 1|||UNK|||It 's|||REQUIRED|||-NONE-|||0
A 3 5|||UNK|||n't it|||REQUIRED|||-NONE-|||0
Target:
It 's Thursday , is n't it ?
The text was updated successfully, but these errors were encountered: