What's changed
- Created Java packaging pipeline and published to Maven repository.
- Added support for conversion of Huggingface FastTokenizer into ONNX custom operator.
- Unified the SentencePiece tokenizer with other Byte Pair Encoding (BPE) based tokenizers.
- Fixed Whisper large model pre-processing bug.
- Enabled eager execution for custom operator and refactored the header file structure.
Contributions
Contributors to ONNX Runtime Extensions include members across teams at Microsoft, along with our community members: @sayanshaw24 @wenbingl @skottmckay @natke @hariharans29 @jslhcl @snnn @kazssym @YUNQIUGUO @souptc @yihonglyu
Contributors
snnn, skottmckay, and 9 other contributors