Fast and memory-efficient library for WordPiece tokenization as it is used by BERT.
nlp
machine-learning
natural-language-processing
ai
tokens
nlp-machine-learning
bert
tokenization
wordpiece
bert-embeddings
wordpiece-tokenization
llm
-
Updated
Jul 2, 2024 - C#