Tokenizer CPU v1.0

A frozen copy of Tokenizer CPU: a stock Hugging Face tokenizer that executes programs, and the tokenizer disagreement it found in real models.

tokcpu-v1.0.zip (sha256 7bb0b46f8cb212038234b9aacca3d1f92544853f0296832148575544bbd879fa) holds:

• the guided incrementer (16 rules): 1011 becomes 1100, 11 to 12, in 8 ticks;
• the universal Brainfuck tokenizer (17,210 rules) with Hello World (154,137 ticks) and Reverse (input stressed, output desserts, 1,904 ticks);
• run.py, a minimal runner on the stock tokenizers library with a step limit, and the expected final tapes, computed from the machines' own rules;
• sigma/, the ΟΔΟΣ disagreement between Python (tokenizers 0.23.3: ids 295 37187 62591) and JavaScript (tokenizers.js 0.2.0: ids 295 42501), from mBERT's tokenizer alone, with no model weights;
• the compatibility results of 2026-10-19, with the runtime versions tested;
• SHA256SUMS for every file.

unzip tokcpu-v1.0.zip && cd tokcpu-v1.0
python -m venv .venv && . .venv/bin/activate
pip install -r requirements.txt
python run.py --check
cd sigma && python sigma.py && npm ci && node sigma.mjs

The same tokenizer files are on the Hugging Face Hub at tag v1.0, commit e833aa9. The dashboard keeps updating; this release does not.