Skip to content

fix(sanitizer): extend UNICODE_BYPASS_RE to cover BIDI, soft hyphen, TAGS block (#4760)#4802

Merged
bug-ops merged 1 commit into
mainfrom
4760-sanitizer-unicode-bypass
Jun 5, 2026
Merged

fix(sanitizer): extend UNICODE_BYPASS_RE to cover BIDI, soft hyphen, TAGS block (#4760)#4802
bug-ops merged 1 commit into
mainfrom
4760-sanitizer-unicode-bypass

Conversation

@bug-ops

@bug-ops bug-ops commented Jun 5, 2026

Copy link
Copy Markdown
Owner

Summary

  • Replaces the 5-codepoint explicit class in UNICODE_BYPASS_RE with \p{Cf} (Unicode Format category) plus explicit U+034F (CGJ, category Mn)
  • Covers BIDI overrides/isolates (U+202A–202E, U+2066–2069), soft hyphen (U+00AD), Mongolian vowel separator (U+180E), invisible math operators, deprecated format chars (U+206A–206F), and the TAGS block (U+E0000–E007F)
  • Adds 9 regression tests for each bypass vector, heterogeneous runs, and false-positive negative
  • No Cargo.toml changes needed — regex crate already enables unicode-perl by default

Security impact

BIDI override characters and the TAGS block are documented in adversarial LLM prompt injection payloads. The old 5-codepoint class left these unblocked; this fix closes the gap for all Unicode Format (Cf) category characters automatically, making the pattern future-proof against new Cf additions.

Residual gap (follow-up)

Variation selectors (U+FE00–FE0F, category Mn) and Hangul fillers (category Lo) are not covered by \p{Cf} and have low practical exploitability in the !…[ context. A separate P3 follow-up issue will track these.

Test plan

  • 328/328 zeph-sanitizer tests pass (cargo nextest run --workspace --features sqlite -E 'package(=zeph-sanitizer)')
  • cargo +nightly fmt --check clean
  • cargo clippy -p zeph-sanitizer -p zeph-memory -p zeph-db -- -D warnings clean
  • Security audit by rust-security-maintenance agent
  • Adversarial critique by rust-critic (verdict: minor, non-blocking)
  • Code review approved

Closes #4760.

@github-actions github-actions Bot added documentation Improvements or additions to documentation rust Rust code changes bug Something isn't working size/M Medium PR (51-200 lines) labels Jun 5, 2026
@bug-ops bug-ops enabled auto-merge (squash) June 5, 2026 12:30
Replace the 5-codepoint explicit class with \p{Cf} (Unicode Format
category) plus explicit U+034F (CGJ, category Mn). This covers BIDI
overrides/isolates (U+202A-202E, U+2066-2069), soft hyphen (U+00AD),
Mongolian vowel separator (U+180E), invisible math operators
(U+2061-2064), deprecated format chars (U+206A-206F), and the TAGS
block (U+E0000-E007F) used in adversarial LLM prompt injection.

Adds 9 regression tests covering each bypass vector, heterogeneous
runs, and a false-positive negative.

Closes #4760.
@bug-ops bug-ops force-pushed the 4760-sanitizer-unicode-bypass branch from 11b816f to bf9e3b2 Compare June 5, 2026 12:30
@bug-ops bug-ops merged commit a487cf1 into main Jun 5, 2026
32 checks passed
@bug-ops bug-ops deleted the 4760-sanitizer-unicode-bypass branch June 5, 2026 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation rust Rust code changes size/M Medium PR (51-200 lines)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security(sanitizer): UNICODE_BYPASS_RE missing BIDI, soft hyphen, and TAGS block codepoints

1 participant