test: strengthen weak assertions in safe_outputs analyzer tests#891
Merged
jamesadevine merged 1 commit intoJun 7, 2026
Merged
Conversation
In aggregate_detection_gate_rejects_all_proposals: - Add summary assertions (proposed_count, not_processed_count, executed_count, rejected_by_execution_count) — the primary metric for detection-gate rejection (not_processed_count) was completely unchecked. - Add finding.category and finding.title assertions — only severity was checked before, meaning a regression in the finding content would have passed silently. In mixed_execution_outcomes_are_rolled_up: - Add rollup.total_rejected assertion — the field was computed but never asserted, leaving its correctness untested. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Test Suite Reduction:
src/audit/analyzers/safe_outputs.rsWhat was wrong
aggregate_detection_gate_rejects_all_proposals: The test never checked thesummarystruct at all — meaningnot_processed_count(the primary metric tracking how many proposals were rejected by the detection gate) was completely unasserted. Additionally, onlyfindings[0].severitywas checked;categoryandtitlewere not verified, so a regression in the finding content would have passed silently.mixed_execution_outcomes_are_rolled_up: Therollupstruct was asserted to exist and have specificby_reasonentries, butrollup.total_rejected(the headline count) was never pinned. A bug intotal_rejected = rejected_by_execution_count + not_processed_countcould go undetected.Changes
aggregate_detection_gate_rejects_all_proposalsmixed_execution_outcomes_are_rolled_uprollup.total_rejectedwas unassertedaggregate_detection_gate_rejects_all_proposalsnow asserts:summary.proposed_count == 2summary.not_processed_count == 2(key gate-rejection metric)summary.executed_count == 0summary.rejected_by_execution_count == 0findings[0].category == "safe_outputs"findings[0].title == "Detection rejected 2 safe output(s)"mixed_execution_outcomes_are_rolled_upnow asserts:rollup.total_rejected == 2Verification
cargo test: all 1766 tests pass ✅cargo clippy --all-targets --all-features: no warnings or errors ✅Warning
Firewall blocked 2 domains
The following domains were blocked by the firewall during workflow execution:
spsprodeus21.vssps.visualstudio.comspsprodweu4.vssps.visualstudio.comSee Network Configuration for more information.