docs: require a scenario for every detection rule - #148
Merged
Conversation
Three of the 16 detection rules shipped without a scenario exercising them: TIMEOUT_NO_HEARTBEAT, FIRMWARE_UPDATE_FAILURE and REPEATED_BOOT_NOTIFICATION. REPEATED_BOOT_NOTIFICATION shows how the gap opens: it landed in #114 and no scenario followed, because nothing in the guide asked for one. States the invariant at the top of "Contributing Detection Rules" and adds it as step 7 of "How to Add a Detection Rule", next to the existing false-positive audit, so both directions of coverage are requested in the same place. Step 6 confirms a rule stays quiet where it should; step 7 confirms it fires where it should. Closes #140
sepehr-safari
added
type:docs
sepehr-safari
added a commit
that referenced
this pull request
Jul 30, 2026
Keeps the living document current after the 0.4.4 release and today's pipeline work, which reached main through the changesets release PR and issue-only activity, neither of which touches this file. Records 0.4.4 (firmware-update-failure, Issue #138, PR #147) and moves the package status table to 0.4.4. Updates the external contribution pipeline: the third good-first-issue completing (#138, the same contributor's second, shipped in 0.4.4), the #140 invariant landing in PR #148, the third external contributor arriving and taking #137, and the patch-not-minor convention holding on its own for 0.4.4. Adds current rule coverage: two of sixteen rules still lack a scenario, and the corpus arithmetic to the v1.0 target now reads 17 plus #108, #137 and #139.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #140.
Three of the 16 detection rules shipped with no scenario exercising them:
TIMEOUT_NO_HEARTBEAT,FIRMWARE_UPDATE_FAILUREandREPEATED_BOOT_NOTIFICATION. A rule with no scenario has no regression guard, sonothing in the corpus catches it breaking.
REPEATED_BOOT_NOTIFICATIONis the worked example: it landed in #114 and noscenario followed, because the "How to Add a Detection Rule" checklist never asked
for one. The final step audits existing scenarios for false positives, which is
the negative half of coverage. The positive half, a scenario whose
expectedFailurescontains the new code, was never requested.Changes
Step 6 confirms a rule stays quiet where it should; step 7 confirms it fires
where it should.
The three uncovered rules are being filled by #137, #138 and #139. This is the
process change that stops a fourth appearing.
Verification
pnpm format:checkandpnpm lintpass. Documentation only; no changeset, sinceCONTRIBUTING.mdis not in the packagefileslist.