Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
1.11 F1 score #15
I tried evaluating a single sentence against itself and I got a Smatch score greater than one (!!), any idea why?
python smatchnew/smatch/smatch.py -f q3.txt q3.txt
@miguelballesteros I think it's because smatch has the assumption that the same triple can only occur once. In your example, you have:ARG0 (p / person :ARG1-of (s / settle-03 :ARG1 p, which results in two same triples <ARG1, settle, person>.
I am not sure if this duplication is a mistake or intended behavior, but we could add something to fix if there are more than one same triples.
If you remove :ARG1 p from your example, the score will be 1.0.