result dismatch with paper

[monkdou0]

the result of SARI and bleu to model is mismatch with the original paper
WHY?????

[louismartin]

Hello @monkdou0
Thanks for the issue.
As mentioned in the the last section of the README, the EASSE implementation fixes bugs and inconsistencies with previous versions of SARI that were used in earlier works.
The scores are not the same although similar.
We highly recommend to use this fixed version and recompute scores using the system outputs provided in the library.

[louismartin]

If you use the latest version of EASSE, you will get these results:

$ easse evaluate -t turkcorpus_test -m sari,bleu -i easse/resources/data/system_outputs/turkcorpus/test/Hybrid
{'bleu': 50.707, 'sari': 31.497}

Papers which use the latest version of SARI include: Multilingual Unsupervised Sentence Simplification.

[monkdou0]

my version is 580bba7,
and the result is
➜ easse evaluate -t turkcorpus_test -m sari,bleu -i easse/resources/data/system_outputs/turkcorpus/test/Hybrid
{'bleu': 50.707, 'sari': 28.154}

[monkdou0]

I have find what's wrong with me
THx! @louismartin

[louismartin]

Hi @monkdou0 do you mind sharing what was wrong with us?
It might help other people in the future!

[monkdou0]

That is totally my own fault.
i change the code of the implementation of SARI function several days ago, and compile it today.
This repo is useful for the task of text simplification.
Very good! thx

[louismartin]

Ok thanks, closing the issue then!