Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add ckpt_path option to LightningModule.test() #2190

Merged
merged 2 commits into from Jun 15, 2020
Merged

Add ckpt_path option to LightningModule.test() #2190

merged 2 commits into from Jun 15, 2020

Conversation

yukw777
Copy link
Contributor

@yukw777 yukw777 commented Jun 15, 2020

Before submitting

  • Was this discussed/approved via a Github issue? (no need for typos and docs improvements)
  • Did you read the contributor guideline, Pull Request section?
  • Did you make sure to update the docs?
  • Did you write any new necessary tests?
  • If you made a notable change (that affects users), did you update the CHANGELOG?

What does this PR do?

Fixes #2046.
If ckpt_path is "best" (default), it loads the best weights saved by ModelCheckpoint for the test loop.
If ckpt_path is a path to a checkpoint file, it loads the weights from the file for the test loop.
If ckpt_path is None, it uses the weights from the end of training for the test loop.
If model parameter is set, ckpt_path is ignored.

PR review

Anyone in the community is free to review the PR once the tests have passed.
If we didn't discuss your PR in Github issues there's a high chance it will not be merged.

Did you have fun?

Make sure you had fun coding 馃檭

@mergify mergify bot requested a review from a team June 15, 2020 03:54
@yukw777
Copy link
Contributor Author

yukw777 commented Jun 15, 2020

@williamFalcon @Borda I finally found some time to work on this, and I saw that you guys have already finalized the changelog entry for 0.8, so I wanted to check with you which version this should go. Obviously, I think it'd be great if we could squeeze this in with 0.8, but I don't want to create unnecessary delays in the release, so let me know!

If ckpt_path is "best" (default), it loads the best weights saved by ModelCheckpoint for the test loop.
If ckpt_path is a path to a checkpoint file, it loads the weights from the file for the test loop.
If ckpt_path is None, it uses the weights from the end of training for the test loop.
If model parameter is set, ckpt_path is ignored.
@Borda Borda added the feature Is an improvement or enhancement label Jun 15, 2020
@Borda Borda added this to the 0.8.0 milestone Jun 15, 2020
@SkafteNicki
Copy link
Member

This is an awesome addition, maybe we should add some documentation about this behavior to:
https://github.com/PyTorchLightning/pytorch-lightning/blob/master/docs/source/test_set.rst

@williamFalcon williamFalcon merged commit 37e7582 into Lightning-AI:master Jun 15, 2020
@yukw777 yukw777 deleted the feature/2046_best-weights branch June 15, 2020 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Is an improvement or enhancement
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Trainer should run the test loop with the best weights when ModelCheckpoint is used
4 participants