Error
Looks like something went wrong!

About

💥 Fast State-of-the-Art Tokenizers optimized for Research and Production

huggingface.co/docs/tokenizers

Apache-2.0 license

Custom properties

Report repository

Releases 1

Packages

Languages

Rust 71.6%
Python 20.2%
Jupyter Notebook 4.6%
TypeScript 2.4%
JavaScript 0.7%
CSS 0.3%
Other 0.2%