Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

I cannot choose [--best_checkpoint_metric BLEU] #1449

Closed
moritay-gm opened this issue Dec 3, 2019 · 1 comment
Closed

I cannot choose [--best_checkpoint_metric BLEU] #1449

moritay-gm opened this issue Dec 3, 2019 · 1 comment

Comments

@moritay-gm
Copy link

In translation task, I have to choice "best_checkpoint_metric".
But, my goal is to get the best BLEU score model in validation dataset.
So, I want to choice BLEU (not "loss") as "best _checkpoint_metric", because the lowest "loss" score model in validation dataset does not necessarily perform the best from the viewpoint of BLEU.

But I cannot choose BLEU as best_checkpoint_metric.

Or, is it natural to use the lowest loss model's BLEU score as metric in NLP conference's paper??
@lematt1991
Copy link
Contributor

Currently there is no way to use BLEU as your validation metric. It is common to use lowest validation loss for choosing checkpoints.

facebook-github-bot pushed a commit that referenced this issue Nov 20, 2020
@myleott @facebook-github-bot
Add fairseq-hydra-train and update docs (#1449)
3b77a61 
Summary: Pull Request resolved: fairinternal/fairseq-py#1449

Test Plan: Imported from OSS

Reviewed By: alexeib

Differential Revision: D25094525

Pulled By: myleott

fbshipit-source-id: 430387d11196d3292933bb168cf09ea16ebc0d3b
sshleifer pushed a commit that referenced this issue Apr 7, 2021
@myleott @facebook-github-bot
Add fairseq-hydra-train and update docs (#1449)
dbfca6e 
Summary: Pull Request resolved: fairinternal/fairseq-py#1449

Test Plan: Imported from OSS

Reviewed By: alexeib

Differential Revision: D25094525

Pulled By: myleott

fbshipit-source-id: 430387d11196d3292933bb168cf09ea16ebc0d3b
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@lematt1991 @moritay-gm