Best model in ModelCheckpoint

[mpariente]

🚀 Feature

Add a best_model attribute in ModelCheckpoint.

Motivation

After training, it would be nice to easily have the path to the checkpoint with the best val loss.

Or is there an argument in the trainer to resume best state after training and I missed it? (totally possible)

[mpariente]

BTW, this is my solution but I don't think this should be necessary

best_k = checkpoint.best_k_models
best_path = [b for b, v in best_k.items() if v == torch.min(best_k.values())][0]

[rohitgr7]

@mpariente Better condition will be:

best_k = checkpoint.best_k_models
best_path = [b for b, v in best_k.items() if v == checkpoint.best)][0]

checkpoint.modecan be max though.

[mpariente]

True, thanks, but this still doesn't change the fact that a best_model_path attribute should be integrated in the checkpoint IMO

[HansBambel]

The save_top_k parameter saves the last k best models. When put to -1 it saves all models. The latest master version also has a save_last parameter that also keeps the latest epoch of the model.
E.g:

checkpoint_callback = ModelCheckpoint(
        filepath=os.getcwd()+"/"+"<your-folder>"+"/{epoch}-{val_loss:.6f}",
        save_top_k=1,
        verbose=False,
        monitor='val_loss',
        mode='min',
        prefix=net.__class__.__name__+"_"
    )

This results in a file called: UNet_epoch=5-val_loss=0.581735.ckpt

[mpariente]

Thanks for the example, I also use it like that.
Still, asking the user to reverse the dict or look for the best loss in the filenames is not super user-friendly. It asks few (not many) additional lines of code which are not necessary, don't you think?

[williamFalcon]

this is already in master.

#1799.

@kepler can you add to docs? (on the checkpoint page)

[mpariente]

Oh ok, sorry for bothering then, I should have checked master
Thanks for the pointer William

[williamFalcon]

it's a great idea :) just someone beat you to it haha