Trying to load bert vocab would result in unpickling error

[wassimseif]

Hey,

I was trying to serve a finetuned bert model. in the logs of torchserve I noticed that the workers are not being loaded.

The exception I got is

2020-04-22 14:06:23,581 [INFO ] W-9004-sentiment-analysis_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle -     self.source_vocab = torch.load(self.manifest['model']['sourceVocab'])
2020-04-22 14:06:23,581 [INFO ] W-9007-sentiment-analysis_1.0-stdout org.pytorch.serve.wlm.WorkerLifeCycle - _pickle.UnpicklingError: invalid load key, '['.

The vocab file I used is this one

Also i noticed that even trying torch.load('vocab.txt') would throw the same error. Am i missing something with how to load the vocab ?

Thank you

[wassimseif]

So i noticed that I'm loading a text vocab file which is not supported?

[wassimseif]

Using a custom handler solved the issue but I would love to know how to get it running with the predefined handlers

[lokeshgupta1975]

@wassimseif custom handlers are not present for all types of models. If you look at https://github.com/pytorch/serve/tree/master/examples, you will find some examples. Over time, we will keep increasing the number of default handlers. Here is the current list of supported default handlers:
https://pytorch.org/serve/default_handlers.html

[fbbradheintz]

If we're going to treat this as a bug (and not a limitation of the built-in handler), we'll need to figure a way to detect whether we're dealing with a parameter file serialized with torch.save(), or a text file - and it needs to not be checking the file extension.

[wassimseif]

Is there a way to save the vocab of a tokenizer as not as text file ?

I'm using the transformers library & it seems that it only saves the vocab as a regular text file

[MFreidank]

@wassimseif For transformers models, you will need to provide custom handler code, the default text_handler does not support transformers models as of now.
For more details on how this would work, see:
https://medium.com/@freidankm_39840/deploy-huggingface-s-bert-to-production-with-pytorch-serve-27b068026d18

[laohur]

@wassimseif For transformers models, you will need to provide custom handler code, the default text_handler does not support transformers models as of now.
For more details on how this would work, see:
https://medium.com/@freidankm_39840/deploy-huggingface-s-bert-to-production-with-pytorch-serve-27b068026d18

what is the benchmark of torchserve for bert?

[vgoklani]

Hey @wassimseif - how did you setup your zschrc profile? it looks amazing!

[wassimseif]

Hey @vgoklani
Thanks !

Oh-My-Zsh + A LOT of configurations & long restless nights

[dhaniram-kshirsagar]

This will be addressed via #302.

[harshbafna]

BERT models are already part of TorchServe examples. Closing.