feat(assert): name the problem instead of printing exit code 127 - #991
Merged
Conversation
Closes #982. assert_true and assert_false ran their argument as a command and reported a bare number when it failed: Expected 'command or function with zero exit code' but got 'exit code: 127' 127 is the shell's not-found code and 126 its not-executable code. Both are now named, with the argument that produced them, so the failure points at its cause. The case that motivated this is not an exotic one. `assert_true "[ -d /tmp ]"` is valid bash that works in an `if`, and it produces exactly this failure, because the argument is run as a command word rather than evaluated. A reader seeing `exit code: 127` has no reason to suspect the shape of their argument and will go and look at /tmp first. assert_false had the worse half of the same problem. It failed only on exit code 0, so 127 counted as "non-zero, therefore false" and a typo in the command name satisfied the assertion: assert_false "definitley_not_a_command" # passed That is a false pass -- the assertion reported success while running nothing. 126 and 127 now fail both assertions, because they mean the command never ran, which is neither true nor false. The wording avoids the literal phrase "command not found" on purpose. runner/diagnostics.sh classifies a test as a runtime error by scanning its output for that exact string, so the obvious phrasing made every one of these failures report as both Failed and Error for a single cause. Found by writing it the obvious way first and watching the duplicate appear. The underlying fragility -- the framework detecting shell errors by string-matching its own output stream -- is filed separately. The docs describe the bracket trap without using brackets: `bashunit doc` strips them while rendering, so the first draft rendered as `assert_true " -d /tmp "` in the CLI, which teaches the wrong lesson. Snapshot regenerated. 1697 sequential / 1656 parallel; baseline + 3, all RED first.
This was referenced Aug 8, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🤔 Background
Related #982
assert_true/assert_falseran their argument as a command and reported a bare number:127 is the shell's not-found code, 126 its not-executable code. Both are now named, with the argument that produced them.
The motivating case isn't exotic —
assert_true "[ -d /tmp ]"is valid bash that works in anif, and produces exactly this, because the argument is run as a command word rather than evaluated. Someone seeingexit code: 127has no reason to suspect their argument's shape and will check/tmpfirst.🐛
assert_falsehad the worse half — a false passIt failed only on exit code 0, so 127 counted as "non-zero, therefore false":
A typo in the command name satisfied the assertion while running nothing. 126 and 127 now fail both assertions, because they mean the command never ran — which is neither true nor false.
runner/diagnostics.shclassifies a test as a runtime error by scanning its output for that exact string. The obvious phrasing made every one of these failures report as bothFailedandErrorfor a single cause.Found by writing it the obvious way first and watching the duplicate appear. The underlying fragility — the framework detecting shell errors by string-matching its own output stream — is worth its own issue and I'll file it.
📖 Docs
The bracket trap is described without using brackets:
bashunit docstrips them while rendering, so the first draft came out asassert_true " -d /tmp "in the CLI, which teaches the wrong lesson. Snapshot regenerated.✅ Verification
3 new tests, all confirmed RED first.
make sa·make lint·bash build.sh bin -v→✅ Build verified ✅· fork budget unchanged · 1697 sequential / 1656 parallel-simple-strict.