Tokenizers integration into onnx models 🤗

[chainyo]

🚀 Feature request

Integrate tokenizers into models while converting them from transformers to onnx format.

Motivation

I use NER camemBERT model for TokenClassification tasks from transformers library that I finetuned to my needs. I converted it to onnx format to deploy it on a Java client where I am planning to use it. But, I would like to integrate the SentenpieceTokenizer I used in python into Java and avoid to recode it in Java.

I found this repo extension to ONNXRuntime package, here
It helps to integrate tokenizers to onnx models by adding one pre-processing layer before model inference, and one post-processing layer after the model inference. It's not so easy to make it works perfectly with my NER model, they only have one GPT2 example and not so much informations in their docs.
I opened an issue here, to describe my problem and to try to make it works.

Your contribution

At this moment, I managed to convert an unsupported model from transformers to onnx by adding its support to transformers library (I could add it via PR). I also managed to adapt transformers NER TokenClassification pipeline to onnx runtime inference + final layer to use inference results, in python.
I am finally trying to understand how I could build a all_in_one_file model in onnx format, and could add examples if I succeed.

Any explanations or help would be much appreciated 🤗

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the contributing guidelines are likely to be ignored.

[thehaseebhassan]

@chainyo Were you able to develop some examples?
I am facing a similar problem: I want Tokenizer and Converted Model in one file.

[ghost]

@chainyo @hacceebhassan
Have either of you found a solution to this? We are now facing the same problem. Too bad this issue was closed, as I think this is a relevant issue. Exporting a single model doesn't do much for me, since I actually want to convert my entire pipeline (preprocessing steps->tokenizer->model), like I can do with a scikit-learn Pipelines...

[beybars1]

Hey, have you managed to solve this issue? I also need to convert HF Tokernizer adn Model into single ONNX file.

[weimeng23]

Hi, any solution here? I meet same problem.

[chainyo]

@chainyo @hacceebhassan Have either of you found a solution to this? We are now facing the same problem. Too bad this issue was closed, as I think this is a relevant issue. Exporting a single model doesn't do much for me, since I actually want to convert my entire pipeline (preprocessing steps->tokenizer->model), like I can do with a scikit-learn Pipelines...

Hey, have you managed to solve this issue? I also need to convert HF Tokernizer adn Model into single ONNX file.

Hi, any solution here? I meet same problem.

Hey guys, unfortunately I haven't worked on any fix on my side, as we decided to re-code the tokenizer in Java in my past company for the task.

Nowadays I don't use onnx anymore, but there is still this package that could help you. Btw 2 years ago I didn't manage to make it work for my use case, but it seems that the package is more mature now, so maybe give it a try?

https://github.com/microsoft/onnxruntime-extensions