BERT Inference Demo using pytorch

[HerbJiang]

Description

Could you provide an jupyter notebook sample for BERT inference demo using pytorch?
Just like current BERT inference demo using mxnet

Who will benefit from this feature?

People already pretrained / fine-tuned their own model using pytorch can easily migrate into this project.

Thanks.

[lanking520]

Currently, we can only do PyTorch Traced model. There are a few examples that may allow us import in DJL:

TorchHub example

thirdparty popular BERT pretrained model

If you are interested, please feel free to search for a BERT pretrained model. You can tested it out using DJL with Jupyter Notebook and help us grow the model zoo! :)

[stu1130]

Hey @HerbJiang,
I have uploaded the HuggingFace Bert Question Answering Model. You can try it out with

git clone https://github.com/awslabs/djl.git
cd djl
./gradlew :examples:run -Dmain=ai.djl.examples.inference.BertQaInference -Dai.djl.default_engine=PyTorch

The source code is here and translator implementation.
I am working on the jupyter notebook, stay tuned

[stu1130]

@HerbJiang
I completed the pytorch BERT QA jupyter notebook.
You can use it with your pretrained BERT along with vocabulary from HuggingFace.
Remember to convert your model to TorchScript.
If you need any help, please let us know.

[stu1130]

feel free to open the issue, if you have any question