Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
BERT fine-tuning with huggingface transformers
Add a notebook that shows what steps need to be taken to fine-tune a BERT model in skorch with the help of the huggingface transformers library. Huggingface transformers is probably the most popular library that implements BERT and other similar models. Ideally, skorch can work with transformers with as little friction as possible. There are actually surprisingly few steps involved, but some of them may not be obvious to most users, which is why the notebook could be helpful. As a cherry on top, the notebook also shows mixed precision training with huggingface accelerate, which indeed results in a very nice boost of training and prediction speed. Other than that, the notebook is actually quite sparse in explanations, instead referring to the fine-tuning docs of hugginface itself.
- Loading branch information