Accelerando-8192

8,192 original glyphs. One reversible tokenizer.

Explore the live site

An angular, mint-green alien alphabet paired with a custom 8,192-entry byte-pair tokenizer. The site includes a searchable character archive, a local browser encoder, an Accelerando specimen, and downloadable fonts, atlases, and reproducible tokenizer data.

• Exactly 8,192 unique glyph shapes.
• Token ID equals glyph ID across the entire vocabulary.
• Custom tokenizer derived from the first 8,192 mergeable ranks of o200k_base, preserving its pretokenization pattern and all 256 byte values. This is not the full OpenAI tokenizer.
• No embeddings or model inference are required. 8,192 is a vocabulary size, not an embedding dimension.
• Glyph font uses private-use Unicode U+F0000–U+F1FFF.
• The Accelerando specimen has 34 lyric lines, 398 token occurrences, and 132 unique tokens, encoded independently per lyric line.

Rotations and reflections may be distinct glyphs. Geometric uniqueness is not a claim of equal human readability.

python3 -m http.server 8080 --directory docs

Open http://localhost:8080. The static site has no build step or backend, and input text never leaves the browser. No analytics or external scripts are loaded.

node tests/tokenizer.test.mjs

The browser tokenizer is compared with reference outputs from the Python tiktoken implementation, including the complete song and multilingual/whitespace cases. The downloadable full package also includes the Python encoder and decoder.

GitHub Pages serves the docs/ directory on the main branch. All asset paths are relative so the site works under a project URL.

• docs/index.html, style.css, app.mjs: site and interactions.
• docs/tokenizer.mjs: browser BPE implementation.
• docs/data/alphabet.json: complete byte vocabulary, glyph geometry, and song token sequence.
• docs/downloads/: full project, four PNGs, vector atlas, tokenizer settings.
• docs/assets/: font, specimen images, and social artwork.

The alien glyph system was inspired by the lettering in the Accelerando music video, using a reference image supplied for this project. Accelerando is by Grimes & Vadakin; its supplied lyrics were transformed into the glyph specimen. Song material remains the property of its respective rights holders. The tokenizer vocabulary is derived from OpenAI tiktoken, which is MIT-licensed; its notice is included in THIRD_PARTY_NOTICES.txt. No ownership of the song is claimed.