Loading a model from 0.4.1 in 0.5.11

[dimidd]

Describe the bug
After saving a model on 5.10 using Classifier.save("my_model.bin"), upgrading to 5.11.
Loading using Classifier.load("my_model.bin") results in KeyError: 'base_model_path'

[madisonmay]

@dimidd thanks again for the bug reports. Going to dig into it -- did not expect this change to break saved model files.

[madisonmay]

Could you post a traceback? This is a very sensical problem to be caused by that upgrade but I'm unable to reproduce.

• I installed 0.5.10.
• Checked to ensure the version is set to 0.5.10
• Trained and saved a simple model
• Installed 0.5.11
• Checked to ensure 0.5.11 was installed
• Was able to successfully load the model

[dimidd]

Sorry, I think it was even an earlier version than 5.10, perhaps 0.5. Is this metadata somehow saved in the model file?

[dimidd]

BTW, I guess the documentation should be updated as well, since it seems that path must be now a dir.
https://github.com/IndicoDataSolutions/finetune/blob/development/finetune/base.py#L373

[dimidd]

P.S. could this change have affected autosave_path? I've tried to use it in 5.11, but nothing was saved.

[madisonmay]

path does not have to be a dir -- the documentation needs to be updated there to reflect the current state of the code, which serializes to a single model file. Sorry for missing that.

[madisonmay]

Digging into the autosave_path report now @dimidd

[madisonmay]

@dimidd OK an update on the autosave_path problem -- that config argument was actually removed in favor of keeping best model parameters in CPU memory. So to get behavior similar to what you had previously:

model = Classifier(keep_best_model=True)
model.fit(data)
model.save(filepath) # this actually writes the saved model parameters out to disk

You can use this in conjunction with the early_stopping_steps argument to exit early if the validation loss fails to decrease for early_stopping_steps batches.

[madisonmay]

@dimidd Seems like there are substantial holes in the documentation for these things and some out of date docs in a few cases. Going to try to block out some time next week to make your life a bit easier in that regard. Thanks for bearing with us!

[dimidd]

Great, thanks a lot! Re the loading issue, perhaps we shouldn't bother with backward compatibility for earlier finetune versions, if there are more pressing issues.

[madisonmay]

@dimidd I think that's wise -- because the project is young we were not too worried about preserving model save files between major finetune versions, but I am sorry we made life hard for you. Hope retraining is not too bad. Thanks again, closing for now!