Skip to content

NLPOptimize/flash-tokenizer

Error
Looks like something went wrong!

About

EFFICIENT AND OPTIMIZED TOKENIZER ENGINE FOR LLM INFERENCE SERVING

github.com/NLPOptimize/flash-tokenizer

Topics

python nlp flash deep-learning cpp tokenizer trie cpp17 bert pybind11 wordpiece huggingface wordpiece-tokenization berttokenizer

Resources

Readme
Activity
Custom properties

Stars

442 stars

Watchers

4 watching

Forks

5 forks
Report repository

Releases 4

dataset Latest
Mar 10, 2025
+ 3 releases

Packages

No packages published

Languages