Pretok is a pre-tokenizer for C-like syntaxes.
-
Updated
Sep 12, 2020 - Rust
Pretok is a pre-tokenizer for C-like syntaxes.
Security Tokens on Lightning Network
CodeChain's official implementation in Rust.
Simple NLP in Rust with Python bindings
ColorBlast is a syntax highlighting library and binary, capable of rendering syntax highlighted code to the terminal, HTML, and images.
Zeetomic is a new generation of trusted global solutions for creating a digital asset. Customer Loyalty, Real Estate, Vested Shares and Payment Solution based on blockchain.
🛥 Vaporetto is a fast and lightweight pointwise prediction based tokenizer. This is a Python wrapper for Vaporetto.
A compiled mecab-ipadic-neologd dictionary for vibrato
3D object recognition tool for WASM. HASH ID sustainable identity creation and its verification.
3D object recognition CLI tool for Linux
The Implementation of 3Dpass Node. Layer 1 decentralized blockchain platform for the tokenization of objects. Proof of Scan is a revolutionary protocol preventing assets form copying. Useful smart-contracts and dApps.
🎤 vibrato: Viterbi-based accelerated tokenizer
🛥 Vaporetto: Very accelerated pointwise prediction based tokenizer
A cli program that calculates the derivative of a function
Sudachi in Rust 🦀 and new generation of SudachiPy
Add a description, image, and links to the tokenization topic page so that developers can more easily learn about it.
To associate your repository with the tokenization topic, visit your repo's landing page and select "manage topics."