Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feature request] allow checkpoint to save partial network #3359

Open
vincentzlt opened this issue Jul 31, 2020 · 1 comment
Open

[feature request] allow checkpoint to save partial network #3359

vincentzlt opened this issue Jul 31, 2020 · 1 comment

Comments

@vincentzlt
Copy link

It would be nice if user can define the weights of partial network that can be saved by checkpoint.

Possible user cases could be GAN network or BERT with various downstream finetune tasks.

@ShawnXuan
Copy link
Contributor

Currently, OneFlow save weights in separate subfolders, and can be removed separately manually. Also we will consider this request to improve the experience of using oneflow checkpoint function.

BTW, OneFlow will try to load weights from user specified checkpoint/model folder, if not exists, weights will be initialized on demand. So, for BERT downstream tasks, we can set pretrain model as the model_load_dir, oneflow will load avaiable weights and init missing weights automatically with Waring for information. see more information in https://github.com/Oneflow-Inc/OneFlow-Benchmark/tree/master/LanguageModeling/BERT.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants