Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BLEU on a scale of 0 to 100 #95

Merged
merged 4 commits into from Feb 26, 2019
Merged

BLEU on a scale of 0 to 100 #95

merged 4 commits into from Feb 26, 2019

Conversation

pmichel31415
Copy link
Contributor

I can't think of any good reason to have BLEU on the 0 to 1 scale. Instead of making the code more complicated by adding an option, I propose we make 0-100 BLEU the default.

@neubig
Copy link
Contributor

neubig commented Feb 21, 2019

Thanks. I think we should at least

  1. be consistent across the various evaluation metrics, such as BLEU, chrF, and METEOR
  2. maybe have a "score_scale" variable somewhere, even if it's a pre-defined constant. That way if we (or a user) want to change it later we can

@pmichel31415
Copy link
Contributor Author

OK I extended this to BLEU, chrF, WER and METEOR. I am less familiar with the other scorers so I don't know the standard, let me know if I should extend to RIBES, ROUGE and the length ratio.

This also adds a --scorer_scale option

@neubig
Copy link
Contributor

neubig commented Feb 22, 2019

Thanks! I think RIBES and ROUGE should also be scaled. Length ratio maybe not?

@pmichel31415
Copy link
Contributor Author

Should be done (and bug free) now

@neubig neubig merged commit 23b156b into master Feb 26, 2019
@neubig neubig deleted the bleu-100 branch July 20, 2019 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants