feat: detect the full hidden-character corpus

[chrisdengso]

Ports the test corpus from out-of-character (MIT) — the most complete public inventory of characters that render as nothing — and closes the gaps it exposed.

Its API is per-character (detect/replace), ours is per-word (analyze), so each case is restated in terms of the invisible signal and normalized output rather than copied literally.

What now gets caught

• Invisible characters that belong to no word — the structural one. A lone format character forms a token with no letters in it, which analyzeToken early-returns on, so anything not touching a letter was unreachable. A separate scan over runs not covered by a word catches them. All four bidi controls in a Trojan Source line are now stripped, not two.
• Blank glyphs that are not whitespace — Hangul fillers, U+2800, musical null notehead. The fillers previously misreported as mixed_script, since they are Hangul letters to the property tables.
• Invisible combining marks — CGJ, Khmer inherent vowels, Kaithi vowel sign. Category Mn, so the \p{Cf} rule missed them. Each stays legitimate inside its own script (Kaithi is now named in scripts.ts for exactly this).
• Stray variation selectors — the ASCII smuggling channel. Registered sequences are untouched.

Not implemented, on purpose

The Unicode whitespace family (U+00A0, U+2000–U+200A, U+3000) — 8 of the 16 blank cases in that corpus. These are White_Space, so every \s matcher already breaks on them (the attacker gains nothing), and they are ordinary typography: NBSP in HTML, U+202F in French, U+3000 in CJK. Flagging them buys false positives and no detection. Recorded as a documented won't-fix in the known-gaps suite.

False positives

A sweep over legitimate multilingual/emoji text caught a real bug during development: ℹ️ (U+2139) is a letter that is also an emoji, so it legitimately takes the presentation selector. The first cut flagged it. That sweep is now a permanent regression suite.

Sequence-building invisibles are exempted only next to Extended_Pictographic, enclosing marks and regional indicators — deliberately not any symbol, so $<ZWSP>100 is still caught (asserted).

Docs

README now leads with the attacks rather than the feature list — Trojan Source, ASCII smuggling (with a snippet that decodes the payload), lookalike domains, filter evasion — plus recipes, a Moderation API section and credits to out-of-character.

Every documented output is asserted by test/readme.test.ts, so the README cannot drift from the library. It earned that immediately: a hand-computed msg.length in the docs was wrong (76 vs 78).

Verification

• 260 tests pass; known-gaps suite fully red as designed (17 remaining, all pre-existing or the deliberate whitespace won't-fix)
• typecheck, lint, format, build, publint, attw all clean
• perf: the new scan uses a Uint8Array mask, not a per-character range search — the existing linear-scaling guard passes

Changeset included (minor → 0.2.0).