Add model weight versioning #76
Comments
|
Just from my side, I think this is a good-idea. It also slightly relates to #71. We can actually integrate this into the model saving and loading once implemented, where some form of 'development' versioning can be defined for local saving. This allows for automatic model checkpointing during training. |
yetinam
added a commit
that referenced
this issue
Apr 4, 2022
This commit adds versioning for model weights. This commit includes the following changes: - Added a weights_version property to SeisBenchModel. - Added a "version" field to json configs of model weights. - json config files are now mandatory for each set of model weights (but using the standard pytorch interface, it is still possible to load weights without config). - The local cache structure now stores models as [name].[json|pt].v[version], i.e., a version suffix was added. Old caches are automatically converted. For compatibility reasons, the required remote cache structure is still downward-compatible. - The list_pretrained function now takes local models into account. - Added list_versions function. - Implemented fine-grained control whether the remote repository or the local cache should be used. - The documentation has been updated to include a remark on versioning. In addition, the control flow of these functions was modified. Tests for all changes were added.
jawooll
pushed a commit
that referenced
this issue
Apr 5, 2022
* Implement model weight versioning (#76) This commit adds versioning for model weights. This commit includes the following changes: - Added a weights_version property to SeisBenchModel. - Added a "version" field to json configs of model weights. - json config files are now mandatory for each set of model weights (but using the standard PyTorch interface, it is still possible to load weights without config). - The local cache structure now stores models as [name].[json|pt].v[version], i.e., a version suffix was added. Old caches are automatically converted. For compatibility reasons, the required remote cache structure is still downward-compatible. - The list_pretrained function now takes local models into account. - Added list_versions function. - Implemented fine-grained control whether the remote repository or the local cache should be used. - The documentation has been updated to include a remark on versioning. In addition, the control flow of these functions was modified. Tests for all changes were added.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
As #73 nicely showed, it might sometimes be desirable to update models weights. I'd therefore suggest to add model weight versioning. My proposal would be to keep all versions available in the repo, but by default download the latest version. Older versions can explicitly be queried using a
versionkwarg. If a version is already cached locally, checks for new versions are only performed when explicitly initiated by the user, for example through an argument likecheck_latest. This way, we avoid that SeisBench needs to "call home" every timefrom_pretrainedis called. I think that would be a privacy issue.#49 proposes a similar addition for datasets. However, I think versioning for models is considerably easier as the file size is much smaller.
The text was updated successfully, but these errors were encountered: