how can I pre-training my own model from the existed model or from scratch

[hischen]

❓ Questions & Help

I want to load the pre-training model like bert offered by google,and train language model on more corpus,how can I do it ?thanks

[LysandreJik]

Hi, you can see how to use the library in the documentation. You might be interested in the library philosophy and the way to load pre-trained models, which is described here. You might also be interested in the examples, which showcase how to fine-tune a language model.

[ravi03071991]

@hischen did you find solution for pre-training BERT on your corpus?
@LysandreJik fine tuning is different from pre-training. I could not find documentation about pre-training the model on a corpus. Can you please help me with that.

Regards,
D. Ravi Theja.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.