🎤 vibrato: Viterbi-based accelerated tokenizer

Sudachi in Rust 🦀 and new generation of SudachiPy

CodeChain's official implementation in Rust.

🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer

Simple NLP in Rust with Python bindings

Painless string scanning.

The Implementation of The Ledger of Things Node. Layer 1 decentralized blockchain platform for the tokenization of objects. Proof of Scan (PoW ASIC resistant + PoS) is a revolutionary protocol preventing assets form copying. Useful smart-contracts and dApps.

🛥 Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto.

Split tokens into word pieces

3D object recognition CLI tool for Linux

A compiled mecab-ipadic-neologd dictionary for vibrato

Basic Tokens Vault

Minimal and fast library for text tokenization.

3D object recognition tool for WASM. HASH ID sustainable identity creation and its verification.

Security Tokens on Lightning Network

A cli program that calculates the derivative of a function

ColorBlast is a syntax highlighting library and binary, capable of rendering syntax highlighted code to the terminal, HTML, and images.

Simple rust port of bpe

Pretok is a pre-tokenizer for C-like syntaxes.

Zeetomic is a new generation of trusted global solutions for creating a digital asset. Customer Loyalty, Real Estate, Vested Shares and Payment Solution based on blockchain.