Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

change Checkpoint callback's save_best_only to save_top_k #70

Closed
vincentzlt opened this issue Aug 8, 2019 · 6 comments · Fixed by #128
Closed

change Checkpoint callback's save_best_only to save_top_k #70

vincentzlt opened this issue Aug 8, 2019 · 6 comments · Fixed by #128
Assignees
@Ir1d
Labels
feature Is an improvement or enhancement help wanted Open to be worked on
Projects
Key features - Roadmap v1.0
Milestone
pl:0.4.x

Comments

@vincentzlt
Copy link

Is your feature request related to a problem? Please describe.
save_best_only is a special case of save_top_k. However, save_tok_k checkpoints can be used to create ensemble model during the test time.

Describe the solution you'd like
keep a dict of {epoch: monitor} of length k, and save new ckeckpoint that can enter this dict, remove the worst checkpoint.
@vincentzlt vincentzlt added feature Is an improvement or enhancement help wanted Open to be worked on labels Aug 8, 2019
@williamFalcon
Copy link
Contributor

williamFalcon commented Aug 8, 2019 •
edited

@vincentzlt good suggestion. I like the change.

Any takers? @vincentzlt interested in making this change?

@williamFalcon williamFalcon modified the milestones: 3.7.1, 0.4.4, 0.4.5 Aug 8, 2019
@Ir1d
Copy link
Contributor

Ir1d commented Aug 13, 2019

i'm interested in this feature, i'll give it a try 😄

@williamFalcon
Copy link
Contributor

awesome!

@Ir1d
Copy link
Contributor

Ir1d commented Aug 16, 2019

Hi, I have some questions.

I noticed save_function option was deleted in this commit: 63d8428

But how does this line work then? https://github.com/williamFalcon/pytorch-lightning/blob/master/pytorch_lightning/callbacks/pt_callbacks.py#L223-L224

Also, there were some save_function=None in docs, which make it somehow confusing.

@williamFalcon
Copy link
Contributor

https://github.com/williamFalcon/pytorch-lightning/blob/master/pytorch_lightning/models/trainer.py#L137

https://github.com/williamFalcon/pytorch-lightning/blob/master/pytorch_lightning/root_module/model_saving.py#L57

Ir1d added a commit to Ir1d/pytorch-lightning that referenced this issue Aug 17, 2019
@Ir1d
feat: change Checkpoint callback's save_best_only to save_top_k
49e35b1 
fix Lightning-AI#70
@Ir1d Ir1d mentioned this issue Aug 17, 2019
Merged
@williamFalcon
Copy link
Contributor

@vincentzlt https://gitter.im/PyTorch-Lightning/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge

@williamFalcon williamFalcon moved this from backlog to 0.4.7 in Key features - Roadmap v1.0 Aug 19, 2019
@williamFalcon williamFalcon moved this from Todo (next release) to In progress in Key features - Roadmap v1.0 Aug 23, 2019
williamFalcon pushed a commit that referenced this issue Aug 24, 2019
@Ir1d @williamFalcon
feat: change Checkpoint callback's save_best_only to save_top_k
582946c 
fix #70
williamFalcon pushed a commit that referenced this issue Nov 19, 2019
@Ir1d @williamFalcon
change Checkpoint callback's save_best_only to save_top_k (#128)
7324dd9 
* docs: enable syntax highlight

* feat: change Checkpoint callback's `save_best_only` to `save_top_k`

fix #70

* docs: update docs for save_top_k

* revert other files

* style: lint for travis-ci

* fix typo

* make flake8 happy

* update according to review

* add tests

* rename func to private

* add doc on `save_top_k == 0`

* make flake8 happy

* update according to PR comments

* change some f-strings

* Update pt_callbacks.py

* Update test_models.py

* update options

* create folders

* Update test_models.py

* change epoch num

* support calling multiple times, add docs and tests

* update docs

* roll back changes in earlystopping

* clean test files

* make flake8 happy

* fix epoch number

* update tests about epoch numbers

* clean debugging code

* fix testing utils codes

* fix testing utils codes

* fix testing utils codes

* fix testing utils codes

* change save_dir to tests/tests according to previous lines

* remove unused overwrite option

* make flake8 happy

* change var name as per review

* make flake8 happy

* update property name to work on master

* elaborate in the docs

* update docs as per review

* revert previous commit

accidentally pressed wrong button when solving conflicts
@nithinivi nithinivi mentioned this issue Jan 27, 2021
Closed
@Borda Borda modified the milestones: 0.4.7, v0.4.x Apr 18, 2021
@radek555 radek555 mentioned this issue Feb 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@Ir1d Ir1d

Labels
feature Is an improvement or enhancement help wanted Open to be worked on
Projects
No open projects
Key features - Roadmap v1.0
  
in Progress
Milestone
pl:0.4.x
Development

Successfully merging a pull request may close this issue.

change Checkpoint callback's save_best_only to save_top_k Ir1d/pytorch-lightning
4 participants
@williamFalcon @Borda @Ir1d @vincentzlt