This is the first official version of Tokenizer as a Python Wheel with a pre-trained model.

!!!Warning!!! The pre-trained tokenizer is only trained on sample text. Please create a text corpus for use.