Harden check_public_safety.py against scanner evasion classes

[next-devin]

Closes #33. Follow-up to the scanner blocklist added in #32 (skills#27).

Hardens scripts/check_public_safety.py against the evasion classes an
adversarial Codex review flagged. Each category lands with a dedicated
regression test, and the scanner's per-line suppression mechanism and
self-non-matching source property are preserved (the full-repo scan stays
clean, verified in CI-style locally).

What changed, by category

1. Separator & word-boundary evasion. Compound token and private-reference
patterns now wrap each body in (?<![A-Za-z0-9]) / (?![A-Za-z0-9]) edge
guards instead of \b (which surrounding underscores defeated), and join their
parts with a broadened [^A-Za-z0-9]* separator. HTML comments are stripped
during normalization. So ., /, +, em-dash, repeated separators, markdown
emphasis, _token_, and injected <!-- --> no longer split a token — while the
edge guards keep ordinary prose (threadbare earth, ecommerce operations)
clean.

2. Cross-line splits. Adjacent lines are rejoined with a single space and
re-scanned for the token and private-reference rules; only matches that actually
span the join are reported (at the first line of the pair). Wrapped allowlisted
URLs do not fabricate a repo name across the break.

3. Unicode normalization. Expanded the invisible-character set (invisible
math operators incl. U+2062, combining grapheme joiner, bidi controls, the
variation-selector supplement U+E0100–U+E01EF, and the deprecated tag block),
and added Unicode NFKC plus Greek/Cyrillic confusable folding so fullwidth and
homoglyph forms fold to ASCII before matching.

4. NUL-byte skip. A NUL alone no longer hides an otherwise-decodable text
file (the previous whole-file evasion). Only genuinely-binary content — bytes
that fail UTF-8 decoding and carry a NUL — is skipped like binary media; other
undecodable files still surface as unreadable.

5. Decode coverage. decode_until_stable now uses unquote_plus, runs more
passes, and additionally resolves JS/JSON \uXXXX/\xHH and CSS \HH escape
sequences.

Scope notes

• Normalization is additive: every rule is still checked against the raw line
  too, so folding can only add detections, never mask one.
• The context-anchored executive/ private reference is intentionally left as-is
  to avoid false positives on the common English word.

Testing

python3 -m unittest discover -s scripts/tests — 32 tests pass (10 new
category tests), and python3 scripts/check_public_safety.py reports no hits on
this branch.

[kilo-code-bot]

Code Review Summary

Status: No Issues Found | Recommendation: Merge

Files Reviewed (2 files)

• scripts/check_public_safety.py
• scripts/tests/test_check_public_safety.py

---

_{Reviewed by minimax-m3 · Input: 48.7K · Output: 1.7K · Cached: 136.7K}