v0.8.1 — per-turn injection works in every writing system
v0.8.0 fixed search, dates, links and chunking — but its verification was CJK-and-Latin shaped, and it never checked the per-turn injection path. Measured properly, that path worked in three of nine languages.
What was broken
Injection runs off terms pulled out of your prompt, and the two patterns doing the pulling recognised ASCII and CJK only.
- Cyrillic, Thai, Devanagari, Arabic, Greek, Hebrew produced no terms at all. Not degraded — the feature never fired once.
- Japanese and Chinese failed the other way: with no spaces to break on, an entire clause became one term, and an entire clause is a literal substring no page contains.
- Vietnamese produced one term: the ASCII pattern cannot start on a letter with a diacritic, so nearly every syllable was dropped. The same fragmentation quietly cost German, French, Spanish, Portuguese, Turkish and Polish their accented words (
überhauptarrived asberhaupt).
Measured, before and after
Fair fixtures — the page and the prompt share the same content words in each language:
| en | ko | ja | zh | ru | th | hi | ar | vi | |
|---|---|---|---|---|---|---|---|---|---|
| before | ✅ | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ |
| after | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ |
All eight unrelated control prompts stay silent, before and after — recall improved without spending precision.
How
- a word pattern for every script, with
\p{M}keeping combining marks attached - runs in scripts written without spaces (Han, Kana, Thai, Lao, Khmer, Myanmar) sliced into word-sized windows.
Script_Extensions, notScript— the prolonged-sound markーendingマイグレーションisScript=Commonand split the run down the middle - floors kept honest: 4 characters for Latin, 3 for a dense script, so the
the/and/forclass still stays out without a stopword list
588 tests, tsc --noEmit clean. No index migration — this release changes only how a prompt is read.