Loading converting PyTorch/ Lightning models

[jimidle]

I could not work this out directly from the code (I have only just started working with this), and I realize that there is not exhaustive documentation and examples (if this project turns out to be good for what I wish to do (train in Python, run via go), then I am willing to contribute. Your README encourages creating issues, I hope that this is the correct place to ask a question.

I am training a model based on a standard Huggingface BERT model. I have trained it and have PyTorch LIghtning checkpoints and of course i can save the model in Pickle vi torch.save(model.state_dict(). "file.pt").

Now, how can i convert that in to the .bin file that this code wants to use. Sorry if this seems obvious, but the converter code I traced seems to want more files than just a state_dict.

Any pointers or example code that I missed.

[jimidle]

I have worked out how to do this from code, once I realized I needed to save the Bert tokenizer. Now I have to explore the tokenizer from the bert model vs the Spago one. But I assume that as the vocab is loaded, it will work as advertised.

If I can use this package, then I will start to contribute, but in case anyone else looks for this, you need to do the following in Python code if you are using Torch lightning and a HuggingFace BERT model (and indeed any trained bert model).

You will have a class that inherits from pytorch_lightning.LightningModule and in your init method, you will be doing something like this:

import pytorch_lightning as pl
class MyClassifier(pl.LightningModule)

    def __init__(self, n_classes=5, steps_per_epoch=None, n_epochs=3, lr=2e-5):
        super().__init__()

        self.bert = BertModel.from_pretrained(BERT_MODEL_NAME, return_dict=True)
       # etc ...

When you are lightning training, you will have almost certainly have set up checkpoints, something like this:

    checkpoint_callback = ModelCheckpoint(
        monitor='val_loss',  # monitored quantity
        filename='tags-{epoch:02d}-{val_loss:.2f}',
        save_top_k=3,  # save the top 3 models
        mode='min',  #  monitored optimization quantity
    )

After training, you get the best model:

        model_path = checkpoint_callback.best_model_path

And you will have used the tokenizer from the pre-trained model:

     tokenizer = BertTokenizer.from_pretrained(BERT_MODEL_NAME)

Now you want to save the model and the vocab etc from your best trained model:

    classy = MyClassifier.load_from_checkpoint(model_path)
    classy.eval()
    tokenizer.save_pretrained('jimdir')
    # Note that we will overwrite the .bin that is saved next - should really just
    # get the config and save that to json directly as that is all we are keeping.
    # Also note, that you may need to modify the config yourself, such as add id2label and
    # label2id if you have a classifier.
    #
    QTmodel.bert.save_pretrained('/Users/jim/.gsl/model')
    # Make sure to save the fine tuned state and not the pre-trained state
    #
    torch.save(MyModel.state_dict(), 'jimdir/pytorch_model.bin')
    

Now, to do the conversion for Spago from this saved pre-trained model, in go code, you can do this (no error handling or params etc, but that's easy):

package bert

import (
	"github.com/nlpodyssey/spago/pkg/nlp/transformers/bert"
)

func ConvertJim() {

    bert.ConvertHuggingFacePreTrained("/path/to/jimdir")
}

You should see:

Serializing model to "/path/to/jimdir/spago_model.bin"... ok
BERT has been converted successfully!

With my model, I get some warnings, such as:

2021/06/29 12:17:17 WARNING!! `cls.predictions.transform.LayerNorm.weight` not initialized
2021/06/29 12:17:17 WARNING!! `classifier.bias` not initialized

But that has nothing to do with this conversion process.