Document the verified CLI interface and both scanner baselines - #6
Merged
Conversation
The docs still described the ThreatCrush flags as unverified and told readers to go read a run log. They have since been verified against the real binary, so that guidance was stale and actively misleading. Replaces it with what was actually observed: the scan command's true signature, the multi-line block output format, and the four parsing details that each caused a real bug (scan-root-relative paths, inconsistent severity bracketing, line :0 on whole-file findings, and Code: excerpts double-counting). Records both baselines side by side. ThreatCrush 15.6% TPR / 0.0% FPR and Semgrep 27.3% / 7.7% are near mirror images — one finds secrets and not code patterns, the other the reverse — which is the argument for a fixed corpus: the numbers compare because the answer key does not move. Also corrects a mislabelled cross-reference: the dead-code-guard caveat was tagged as PRD Open Question 1, which actually asked about PR-level scanning. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
ThreatCrush scan results9 finding(s) from
Detection coverage — FAIL
Findings
Every file under Posted by |
ralyodio
added a commit
that referenced
this pull request
Aug 1, 2026
Replaces the stale "flags are unverified" guidance with the observed interface and output format. Records ThreatCrush at 15.6% TPR / 0.0% FPR and Semgrep at 27.3% / 7.7%. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The docs still called the ThreatCrush flags "unverified" and pointed readers at a run log. They're verified now, so that was stale and misleading.
Replaces it with what was actually observed — the real
scansignature, the block output format, and the four parsing details that each caused a bug (scan-root-relative paths, inconsistent severity bracketing, line:0,Code:double-counting).Adds both baselines to the README:
Near mirror images — one finds secrets, the other code patterns.
Also fixes a mislabelled cross-reference (the dead-code-guard caveat was tagged as Open Question 1, which was actually about PR-level scanning).
🤖 Generated with Claude Code