fix(sanitizer): extend UNICODE_BYPASS_RE to cover BIDI, soft hyphen, TAGS block (#4760)#4802
Merged
Merged
Conversation
Replace the 5-codepoint explicit class with \p{Cf} (Unicode Format
category) plus explicit U+034F (CGJ, category Mn). This covers BIDI
overrides/isolates (U+202A-202E, U+2066-2069), soft hyphen (U+00AD),
Mongolian vowel separator (U+180E), invisible math operators
(U+2061-2064), deprecated format chars (U+206A-206F), and the TAGS
block (U+E0000-E007F) used in adversarial LLM prompt injection.
Adds 9 regression tests covering each bypass vector, heterogeneous
runs, and a false-positive negative.
Closes #4760.
11b816f to
bf9e3b2
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
UNICODE_BYPASS_REwith\p{Cf}(Unicode Format category) plus explicit U+034F (CGJ, category Mn)regexcrate already enablesunicode-perlby defaultSecurity impact
BIDI override characters and the TAGS block are documented in adversarial LLM prompt injection payloads. The old 5-codepoint class left these unblocked; this fix closes the gap for all Unicode Format (
Cf) category characters automatically, making the pattern future-proof against newCfadditions.Residual gap (follow-up)
Variation selectors (U+FE00–FE0F, category Mn) and Hangul fillers (category Lo) are not covered by
\p{Cf}and have low practical exploitability in the!…[context. A separate P3 follow-up issue will track these.Test plan
zeph-sanitizertests pass (cargo nextest run --workspace --features sqlite -E 'package(=zeph-sanitizer)')cargo +nightly fmt --checkcleancargo clippy -p zeph-sanitizer -p zeph-memory -p zeph-db -- -D warningscleanrust-security-maintenanceagentrust-critic(verdict: minor, non-blocking)Closes #4760.