Skip to content

feat: add pipe mode and composable --strip/--halt action flags#13

Merged
mit-d merged 3 commits intomainfrom
feat/pipe-mode-action-flags
Mar 29, 2026
Merged

feat: add pipe mode and composable --strip/--halt action flags#13
mit-d merged 3 commits intomainfrom
feat/pipe-mode-action-flags

Conversation

@mit-d
Copy link
Copy Markdown
Owner

@mit-d mit-d commented Mar 29, 2026

Summary

  • Pipe mode (check-unicode -): reads stdin line-by-line and writes to stdout, enabling use as a streaming Unix filter for log monitoring, CI pipelines, and editor buffer filtering
  • --strip [dangerous|all]: removes non-ASCII characters from output; dangerous strips only invisible/bidi chars, all (default) strips any remaining non-ASCII after allow-list processing
  • --halt [dangerous|all]: stops immediately on first matching character; dangerous (default) halts on invisible/bidi chars, all halts on any non-ASCII
  • All three action flags (--fix, --strip, --halt) are fully composable and work identically across file and pipe modes

Examples

check-unicode - < file.txt                    # check stdin, passthrough to stdout
check-unicode --fix --strip dangerous -       # fix fixable, strip bidi attacks
check-unicode --halt - < input.txt            # halt on first dangerous char
tail -f app.log | check-unicode --fix -       # stream-filter logs

Design

Processing order per character: allow-list → fix → halt → strip

  • Pipe mode streams line-by-line with per-line findings on stderr and flushed stdout
  • --halt stops mid-file/mid-stream — the triggering file is never modified
  • Allow-lists (including per-file overrides) are respected by all flags
  • Spec: docs/specs/2026-03-29-pipe-mode-and-action-flags-design.md

Test plan

  • 333 tests passing (68 new), covering all flag combinations in both file and pipe modes
  • Parametrized integration tests for flag interactions with config files, overrides, severity, --quiet, --no-color
  • ruff, ty, and all pre-commit hooks clean
  • Manual smoke tests with piped input (--fix, --strip, --halt combinations)

mit-d added 2 commits March 29, 2026 12:48
Pipe mode (`check-unicode -`) reads stdin line-by-line and writes to
stdout, enabling use as a streaming Unix filter for log monitoring, CI
pipelines, and editor buffer filtering. Findings are emitted to stderr
per-line with full context display (rendered source line, caret markers,
codepoint details).

New composable action flags work identically across file and pipe modes:

- `--strip [dangerous|all]` removes non-ASCII characters from output.
  Default: all. Respects allow-lists.
- `--halt [dangerous|all]` stops immediately on first matching character.
  Default: dangerous.
- `--fix`, `--strip`, and `--halt` are fully composable. Processing
  order: allow-list -> fix -> halt -> strip.

Examples:
  check-unicode - < file.txt
  check-unicode --fix --strip dangerous -
  check-unicode --halt - < input.txt
  cat logs | check-unicode --fix --strip dangerous -
@mit-d mit-d force-pushed the feat/pipe-mode-action-flags branch from 13c0767 to d044dc1 Compare March 29, 2026 18:55
@mit-d mit-d merged commit 56fd8e5 into main Mar 29, 2026
4 checks passed
@mit-d mit-d deleted the feat/pipe-mode-action-flags branch March 29, 2026 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant