Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

如何载入保存训练好的模型? #52

Closed
Ryan906k9 opened this issue Mar 2, 2021 · 4 comments
Closed

如何载入保存训练好的模型? #52

Ryan906k9 opened this issue Mar 2, 2021 · 4 comments
Assignees

Comments

@Ryan906k9
Copy link

使用了预训练模型进行 finetune:

import paddlenlp as ppnlp

# 设置想要使用模型的名称
MODEL_NAME = "ernie-1.0"

ernie_model = ppnlp.transformers.ErnieModel.from_pretrained(MODEL_NAME)
model = ppnlp.transformers.ErnieForTokenClassification.from_pretrained(MODEL_NAME, num_classes=train_ds.num_label)
tokenizer = ppnlp.transformers.ErnieTokenizer.from_pretrained(MODEL_NAME)

训练完成后:

model.save_pretrained('./checkpoint')
tokenizer.save_pretrained('./checkpoint')

请问训练完成后应该如何加载保存的模型?

@guoshengCS
Copy link
Collaborator

model.save_pretrained('./checkpoint')tokenizer.save_pretrained('./checkpoint')对应,使用model.from_pretrained('./checkpoint')tokenizer.from_pretrained('./checkpoint')就可以了

@guoshengCS
Copy link
Collaborator

model.from_pretrainedtokenizer.from_pretrained是支持按照bert-base-uncased这种模型名称或者本地训练模型的路径来导入模型的

@guoshengCS
Copy link
Collaborator

另外通过

model.save_pretrained('./checkpoint')
tokenizer.save_pretrained('./checkpoint')

保存下来的内容(如下)中的model_state.pdparams是通过Paddle原生的save接口保存的,也可以使用Paddle原生的load接口来加载

-rw-r--r-- 1 root root  416 Mar  2 07:56 model_config.json
-rw-r--r-- 1 root root 632M Mar  2 07:56 model_state.pdparams
-rw-r--r-- 1 root root  141 Mar  2 07:56 tokenizer_config.json
-rw-r--r-- 1 root root 227K Mar  2 07:56 vocab.txt

@Ryan906k9
Copy link
Author

另外通过

model.save_pretrained('./checkpoint')
tokenizer.save_pretrained('./checkpoint')

保存下来的内容(如下)中的model_state.pdparams是通过Paddle原生的save接口保存的,也可以使用Paddle原生的load接口来加载

-rw-r--r-- 1 root root  416 Mar  2 07:56 model_config.json
-rw-r--r-- 1 root root 632M Mar  2 07:56 model_state.pdparams
-rw-r--r-- 1 root root  141 Mar  2 07:56 tokenizer_config.json
-rw-r--r-- 1 root root 227K Mar  2 07:56 vocab.txt

谢谢!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants