New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Trainer should run the test loop with the best weights when ModelCheckpoint is used #2046
Comments
Something like this? |
I would make the |
Yeah this should definitely be the default behavior. Another question is, can we only do this when |
why not make it:
with the option for a string ‘best’ and make this the default |
very good and |
Nice! I like that idea. To summarize:
|
ummm. i prefer None to disable it. |
ah yeah since using the last epoch weights is the current behavior, setting it to |
current behavior is equivalent to None new default behavior should be “best” |
🚀 Feature
Motivation
I noticed that even when
ModelCheckpoint
is used,Trainer
by default runs the test loop with the last weights, not the best weights saved byModelCheckpoint
. I believe the sensible default here is to run the test loop with the best weights saved byModelCheckpoint
.Pitch
Now that
ModelCheckpoint
has a pointer to the best weights,Trainer
can replace the last weights with the best weights before running the test loop automatically.Alternatives
Possibly, this could be another option to
Trainer
. I don't like this as much b/c this is the behavior most users would expect.Additional context
The text was updated successfully, but these errors were encountered: