fix(fuzz): strip real-shell error lines from stderr before banned-shape check#1623
Merged
Conversation
…pe check
When a fuzz/proptest target inlines arbitrary input bytes into a shell
script, bash and ls produce error messages that quote the input verbatim:
bash: <cmd>: command not found
bash: <path>: No such file or directory
ls: cannot access '<path>': No such file or directory
These echoes can accidentally form a banned substring even when no
internal Debug formatter ran. Two real cases:
- arithmetic_fuzz / glob_fuzz: input contained `/.rustup/toolchains/`
literally; bash echoed it back. Filtered at the input layer in #1621
and the arithmetic_fuzz follow-up.
- glob_fuzz (new): input ended with `Tok"`. Bash treated `Tok:` as a
command and rendered `bash: Tok:: command not found` — the trailing
`:` chrome from the error format glued onto the input's final `:` to
form the banned parser-token shape `Tok::`. Pre-filtering the input
for `Tok::` doesn't catch this — the substring only forms after
bash's formatter runs.
Structural fix: in `assert_fuzz_invariants`, strip lines that match a
recognized real-shell error template before running the banned-shape
check. The byte-length cap (`MAX_STDERR_BYTES`) and the host-canary
check (TM-INF-013) still run on the unfiltered stderr, so flood and
env-leak regressions are still caught. The strict `assert_no_leak`
path (used by per-builtin tests) is unchanged — non-fuzz tests must
not produce shell echoes in the first place.
Recognized templates are conservative: prefix `bash: ` or `ls: ` plus
a known suffix (`: command not found`, `: No such file or directory`,
`: Is a directory`, `: Permission denied`, `: cannot execute …`, plus
the `command not found. Did you mean: ., :, [?` variant). Lines that
look similar but don't match the exact template stay in stderr, so
real Debug leaks that happen to coexist with shell errors still trip
the assertion.
Tests: 8 unit tests in `testing::tests` cover both directions —
strips known shell echoes, keeps internal panic/Debug lines, keeps
partial matches and lines from other tools. Threat-model TM-INF-022
section updated to document the carve-out.
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
bashkit | ca39280 | Commit Preview URL Branch Preview URL |
May 10 2026, 09:51 AM |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a fuzz/proptest target inlines arbitrary input bytes into a shell script, bash and ls produce error messages that quote the input verbatim:
These echoes can accidentally form a banned substring even when no internal Debug formatter ran. Two real cases:
/.rustup/toolchains/literally; bash echoed it back. Filtered at the input layer in fix(fuzz): drop glob_fuzz inputs that contain banned debug shapes #1621 and fix(fuzz): drop arithmetic_fuzz inputs that contain banned debug shapes #1622.Tok". Bash treatedTok:as a command and renderedbash: Tok:: command not found— the trailing:chrome from bash's error format glued onto the input's final:to form the banned parser-token shapeTok::. Pre-filtering the input forTok::doesn't catch this — the substring only forms after bash's formatter runs.Fix
Structural fix in
assert_fuzz_invariants: strip lines that match a recognized real-shell error template before running the banned-shape check.MAX_STDERR_BYTES) still runs on the unfiltered stderr — flood regressions still caught.assert_no_leakpath (used by per-builtin tests) is unchanged — non-fuzz tests must not produce shell echoes in the first place.Recognized templates are conservative: prefix
bash:orls:plus a known suffix. Lines that look similar but don't match the exact template stay in stderr, so real Debug leaks that happen to coexist with shell errors still trip the assertion.Test plan
testing::testscover both directions — strip known shell echoes, keep internal panic/Debug lines, keep partial matches and lines from other tools.cargo test -p bashkit --lib testing::greencargo test -p bashkit --test proptest_security --all-featuresgreen (18 cases)cargo clippy -p bashkit --lib --tests -- -D warningscleancargo fmt --checkcleanFuzz Testingon this branch and confirmglob_fuzzjob is green