High performance Chinese tokenizer with both GBK and UTF-8 charset support based on MMSEG algorithm developed by ANSI C. Completely based on modular implementation and can be easily embedded in other programs, like: MySQL, PostgreSQL, PHP, etc.

C library for Lexical Analysis

🔋 In-place lightweight JSON parser

A cross-platform easy-to-use programming language inspired by AutoIt (BASIC-like).

🧪 An imperative, dynamically-typed, interpreted, general-purpose programming language

🔋 In-place lightweight XML parser

Tiny, zero-allocation JSON tokenizer written in ANSI C

A set of libraries that allow you to manipulate files, linked lists, sorting algorithms and much more.

Colossal compiler to T++ language with the better memory manager to the Compiler discipline - Compiler Construction Principles and Practice by Kenneth C. Louden

A natural language tokenizer

X86-64 bilateral instruction tokenizer implemented in C. Supports the following processor extensions: AES, AVX, AVX2, AVX512, FMA, MMX, SSE, SSE2, SSE3, SSE4, x87(FPU), VMX. In order to ease testing, a diassembler which transforms tokens into compilable assembly (for NASM compiler) has been implemented.

An ngram tokenizer for sqlite3 fts5

A lexer for mathematical expressions in C

Url Token Authorization for Nginx

My first attemp to write a sqlite3 clone. It's just an experimental attempt to understand RDBMS. Never use it in any production.

A basic, yet a powerful implementation of shell in C for unix/linux systems

An (incomplete) interpreted scripting language from scratch in C.

A fast sentence/word tokenizer, and punctuation remover.

ragel token scanner for nginx conf like config file with lemon parser.

Creating a simple shell in C.