feat: --fail-on, --quiet, exit code 2, rules and explain (#25) - #76
Merged
Conversation
The CLI was one `check` subcommand with a binary exit code, so a team could not
adopt the scanner incrementally or tune what blocks their build.
Exit codes are this tool's real interface to CI, and there are now three:
0 no findings
1 findings at or above --fail-on
2 findings, all below --fail-on
The third is the one that matters. Without it `--fail-on critical` would exit 0
on a file full of HIGH findings, and any pipeline checking for zero would call
it clean — the flag would quietly become a way to hide things rather than a way
to prioritise them. Matches the convention spec-linter already uses here.
`--fail-on` raises the bar for FAILING, not for REPORTING. Findings below it are
still printed, because a user who cannot see them cannot judge whether the bar
is set right. Pinned by a test.
`rules` and `explain <PI0XX>` answer the question a blocked commit creates.
Both report the EFFECTIVE severity — resolved against the category default —
since a listing showing a blank for every pattern that inherits would be worse
than no listing. `explain` is case-insensitive, ends with the exact suppression
directive for that pattern, and on an unknown id suggests nearby ones rather
than just failing.
Two bugs worth naming:
- `{:<9}` silently did nothing to the severity column. A width specifier only
pads a custom Display impl if that impl routes through `Formatter::pad`, and
this one does not, so every column ran together. Rendered to a String first.
- The new items landed between `#[derive(Subcommand)]` and `enum Commands`,
which quietly reattached the derive to the wrong type. Every `#[arg]` in the
file then failed to resolve.
Depended on #21: with everything graded CRITICAL or HIGH there was nothing for
`--fail-on` to threshold on, which is why that issue lists this as blocked.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Covers CLI-06 and CLI-07 from #25. Depended on #21 — with everything graded CRITICAL or HIGH there was nothing for
--fail-onto threshold on, which is why that issue lists this as blocked.Exit codes are the real interface to CI
The third one is the point. Without it,
--fail-on criticalwould exit0on a file full of HIGH findings, and any pipeline checking for zero would call it clean — the flag would quietly become a way to hide things rather than a way to prioritise them.Verified as a matrix:
--fail-on criticalhighmediumlow--fail-onraises the bar for failing, not for reporting. Findings below it are still printed, because a user who can't see them can't judge whether the bar is set right. Pinned by its own test.rulesandexplainThe question a blocked commit creates:
Both show the effective severity, resolved against the category default — a listing with a blank for every pattern that inherits would be worse than none.
explainis case-insensitive, ends with the exact suppression directive, and on an unknown id suggests nearby ones instead of just failing.rules --format jsonis machine-readable.Two bugs worth naming
{:<9}silently did nothing to the severity column. A width specifier only pads a customDisplayimpl if that impl routes throughFormatter::pad— this one doesn't, so every column ran together. Rendered to aStringfirst.The new items landed between
#[derive(Subcommand)]andenum Commands, quietly reattaching the derive to the wrong type. Every#[arg]in the file then failed to resolve, with an error pointing at lines I hadn't touched.Tests
7 new, including the full exit-code matrix and the below-threshold-still-reported guarantee. Full suite 23 binaries green, clippy clean.
Not in this PR:
--baseline(CLI-08),--no-color,--stdin-name.--baselineis the larger adoption feature and deserves its own change.