fix(ci): the invisible-character gate never matched anything - #61
Conversation
MEASURED 2026-08-27: this gate's pattern caught 0 OF 6 invisible-character test
cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi
override or word joiner.
ROOT CAUSE: the pattern used UTF-8 BYTE sequences (\xc2\xa0) while grep -P
matches CHARACTERS. Bytes c2 a0 are ONE character U+00A0; \xc2\xa0 asks for TWO
characters, U+00C2 then U+00A0, which is never present.
grep -P '\xc2\xa0' -> miss
grep -P '\x{a0}' -> MATCH
Only \x00 worked, being single-byte in both readings.
FIXED: codepoint escapes; C0 control characters \x01-\x08,\x0B,\x0C,\x0E-\x1F
added (TAB/LF/CR excluded); and grep -a, without which grep skips any NUL-bearing
file as binary.
The C0 range matters: a stray BACKSPACE byte made a workflow unparseable in
developer-ecosystem, so it never ran, and this linter called it clean.
Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
VERIFIED: YAML re-parsed, and the corrected pattern was confirmed to catch a real
NBSP before the change was kept.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
Included review availability: Your plan provides up to 1 included review per hour; 0 remain after this review. 📜 Recent review details⏰ Context from checks skipped due to timeout. (26)
🔇 Additional comments (1)
📝 WalkthroughSummary by CodeRabbit
WalkthroughThe dogfood gate now detects a wider set of invisible characters using Unicode code-point escapes. It also scans binary files as text. ChangesInvisible-character gate
Estimated code review effort: 1 (Trivial) | ~5 minutes Merge Risk: ⚪ Minimal · up to This localized workflow pattern change is merge-ready after normal checks; no actionable merge-blocking risk remains. Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Full details: Description checkExplanation The description explains the root cause, the code changes, and verification results. It is on topic and mostly complete, although it does not use the template headings or include the checklist and testing sections explicitly. Full details: Linked Issues checkExplanation The change satisfies the codepoint escape, C0 control range, and grep -a requirements in issue Resolution Add the separate byte-wise leading-BOM check, update stdlib/ByteDetector.affine and config.ncl to keep compiled-linter behaviour aligned, and apply the correction to all other inlined copies required by issue Full details: Docstring CoverageExplanation No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0 files. (1 skipped: 1 unsupported.)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Up to standards ✅🟢 Issues
|
![codacy-production[bot]](https://avatars.githubusercontent.com/in/56611?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
This PR successfully addresses the non-functional invisible-character gate by transitioning to PCRE-compatible Unicode escapes and incorporating the -a flag to prevent skipping files containing null bytes. The code is up to standards according to Codacy analysis.
However, there are opportunities to improve the efficiency of the search command and the robustness of the CI gate itself. The current implementation spawns a process for every file and suppresses potential errors that could indicate environment issues. Additionally, the lack of test fixtures or automated validation for these patterns leaves the gate at risk of future regressions.
About this PR
- No automated tests or fixture files containing the targeted invisible characters (e.g., BOM, ZWSP, or C0 control characters) were added to the repository. Consider adding a 'test-fixtures' directory with samples of these characters to verify the regex patterns and prevent regressions.
Test suggestions
- Detect Non-breaking space (U+00A0) using the updated codepoint escape
- Detect C0 control characters (e.g., \x08 backspace) while ignoring TAB (\x09)
- Verify grep -a correctly processes and scans files containing null bytes (\x00)
- Detect Zero-width characters (U+200B, U+200C, U+200D) and BOM (U+FEFF)
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Detect Non-breaking space (U+00A0) using the updated codepoint escape
2. Detect C0 control characters (e.g., \x08 backspace) while ignoring TAB (\x09)
3. Verify grep -a correctly processes and scans files containing null bytes (\x00)
4. Detect Zero-width characters (U+200B, U+200C, U+200D) and BOM (U+FEFF)
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| -o -name '*.idr' -o -name '*.zig' -o -name '*.v' -o -name '*.jl' \ | ||
| -o -name '*.gleam' -o -name '*.hs' -o -name '*.ml' -o -name '*.sh' \) \ | ||
| -exec grep -Prl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | ||
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null |
There was a problem hiding this comment.
⚪ LOW RISK
Suggestion: The search process is inefficient and contains redundant flags. Spawning a new grep process for every file using \; is slow; changing this to + allows find to pass multiple files to a single process. Additionally, the -r (recursive) flag is redundant because find already handles recursion, and redirecting stderr to /dev/null is discouraged as it masks potential issues such as missing PCRE support in the runner.
| -exec grep -aPrl "$PATTERNS" {} \; > /tmp/empty-lint-results.txt 2>/dev/null | |
| -exec grep -aPl "$PATTERNS" {} + > /tmp/empty-lint-results.txt |
1 participant
Measured 2026-08-27: this gate caught 0 of 6 invisible-character test cases. It has never detected an NBSP, zero-width space, BOM, soft hyphen, bidi override or word joiner.
Root cause
The pattern used UTF-8 byte sequences (
\xc2\xa0) whilegrep -Pmatches characters. Bytesc2 a0are one character U+00A0;\xc2\xa0asks for two, U+00C2 then U+00A0 — never present.Only
\x00worked, being single-byte in both readings. The gate ran, passed, and could not see what it exists to see.Fixed
\x01-\x08,\x0B,\x0C,\x0E-\x1Fadded (TAB/LF/CR excluded)grep -a— without it grep skips any NUL-bearing file as binaryThe C0 range matters: a stray backspace byte made a workflow unparseable in
developer-ecosystem, so it never ran — and this linter called it clean.Canonical fix: hyperpolymath/empty-linter#70. 1 file(s) here.
Verified: YAML re-parsed, and the corrected pattern was confirmed to catch a real NBSP before the change was kept.