BERT fine-tuning with huggingface transformers (#839)

BERT fine-tuning with huggingface transformers

Add a notebook that shows what steps need to be taken to fine-tune a
BERT model in skorch with the help of the huggingface transformers
library.

Huggingface transformers is probably the most popular library that
implements BERT and other similar models. Ideally, skorch can work with
transformers with as little friction as possible.

There are actually surprisingly few steps involved, but some of them may
not be obvious to most users, which is why the notebook could be
helpful.

As a cherry on top, the notebook also shows mixed precision training
with huggingface accelerate, which indeed results in a very nice boost
of training and prediction speed.

Other than that, the notebook is actually quite sparse in explanations,
instead referring to the fine-tuning docs of hugginface itself.

docs/requirements.txt

@@ -1,6 +1,8 @@
 docutils<0.18
+gpytorch>=1.5
 numpy>=1.13.3
 torch>=1.11.0
+transformers
 sphinx-rtd-theme==0.4.0
 numpydoc==0.8.0
 scikit-learn>=0.19.1

docs/skorch.rst

@@ -9,6 +9,7 @@ skorch
    dataset
    exceptions
    helper
+   hf
    history
    net
    probabilistic

docs/user/customization.rst

@@ -191,9 +191,10 @@ modules, criteria, and optimizers? You have to follow these rules:
 3. Their names should end on an underscore. This is true for all attributes that
    are created during ``initialize`` and distinguishes them from arguments
    passed to ``__init__``. So a name for a custom module could be ``mymodule_``.
-4. Inside the initialization method, use :meth:`.get_params_for` (or,
-   if dealing with an optimizer, :meth:`.get_params_for_optimizer`) to
-   retrieve the arguments for the constructor of the instance.
+4. Inside the initialization method, use
+   :meth:`skorch.net.NeuralNet.get_params_for` (or, if dealing with an
+   optimizer, :meth:`skorch.net.NeuralNet.get_params_for_optimizer`) to retrieve
+   the arguments for the constructor of the instance.
 
 Here is an example of how this could look like in practice:
 

docs/user/tutorials.rst

@@ -23,4 +23,6 @@ The following are examples and notebooks on how to use skorch.
 
 * `Advanced Usage <https://nbviewer.jupyter.org/github/skorch-dev/skorch/blob/master/notebooks/Advanced_Usage.ipynb>`_ - Dives deep into the inner works of skorch. `Run in Google Colab 💻 <https://colab.research.google.com/github/skorch-dev/skorch/blob/master/notebooks/Advanced_Usage.ipynb>`_
 
-* `Gaussian Processes <https://nbviewer.jupyter.org/github/skorch-dev/skorch/blob/master/notebooks/Gaussian_Processes.ipynb>`_ - Train Gaussian Processes with the help of GPyTorch `Run in Google Colab 💻 <https://colab.research.google.com/github/skorch-dev/skorch/blob/master/notebooks/Gaussian_Processes.ipynb>`_
+* `Gaussian Processes <https://nbviewer.jupyter.org/github/skorch-dev/skorch/blob/master/notebooks/Gaussian_Processes.ipynb>`_ - Train Gaussian Processes with the help of GPyTorch. `Run in Google Colab 💻 <https://colab.research.google.com/github/skorch-dev/skorch/blob/master/notebooks/Gaussian_Processes.ipynb>`_
+
+* `Huggingface Finetunging <https://nbviewer.jupyter.org/github/skorch-dev/skorch/blob/master/notebooks/Huggingface_Finetuning.ipynb>`_ - Fine-tune a BERT model for text classification with the huggingface transformers library and skorch.  `Run in Google Colab 💻 <https://colab.research.google.com/github/skorch-dev/skorch/blob/master/notebooks/Huggingface_Finetuning.ipynb>`_