feat(investigate): configurable dismissable severity ceiling - #147
Conversation
There was a problem hiding this comment.
Pull request overview
Adds a configurable severity ceiling for automatic Dependabot alert dismissal while preserving confidence safeguards.
Changes:
- Adds
dismiss_max_severity, defaulting tomedium. - Enforces severity ranking and safe fallback behavior.
- Adds coverage for raised ceilings, invalid values, and unknown severities.
Reviewed changes
Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
config/defaults.yml |
Defines the default ceiling. |
.github/actions/setup-target/action.yml |
Reads and exposes the setting. |
.github/workflows/investigate-security-alert.yml |
Passes the setting to remediation. |
scripts/github_utils.py |
Centralizes severity rankings. |
scripts/sweep.py |
Uses the shared ranking. |
scripts/post_alert_action.py |
Enforces the configurable ceiling. |
tests/scripts/test_post_alert_action.py |
Tests dismissal-ceiling behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
8191846 to
fc16926
Compare
Replace the hardcoded high/critical dismissal block with investigate.dismiss_max_severity (low|medium|high|critical, default medium). An alert is dismissed only when severity <= the ceiling AND confidence meets the floor (#145), so confidence is the safety rail and severity is the opt-in scope — a repo confident it's unaffected can raise the ceiling to critical. Unknown/invalid values fail safe; the skip reason (above ceiling / needs manual review / below required confidence) is surfaced in the step summary. SEVERITY_RANK is shared from github_utils. Stacked on #145. Closes #144.
fc16926 to
2c9125f
Compare
| # (a confidence floor for every dismissal; which severities are dismissable is | ||
| # separate) |
There was a problem hiding this comment.
question (non-blocking): This is a nice little clean-up. I'm just curious if an LLM cleaned this up on its own or did you have to prompt it or manually remove this? I don't often see LLMs remove code.
There was a problem hiding this comment.
I prompted AI to change it here: #147 (comment)
Still a ways to go before this happens on its own eh? 😆
| from scripts.config_utils import load_repo_config | ||
| from scripts.github_utils import has_codeowner_approval, is_bot | ||
| from scripts.github_utils import SEVERITY_RANK, has_codeowner_approval, is_bot | ||
| except ImportError: | ||
| from config_utils import load_repo_config # type: ignore[no-redef] | ||
| from github_utils import has_codeowner_approval, is_bot # type: ignore[no-redef] | ||
| from github_utils import ( # type: ignore[no-redef] | ||
| SEVERITY_RANK, | ||
| has_codeowner_approval, | ||
| is_bot, | ||
| ) |
There was a problem hiding this comment.
suggestion (non-blocking): not something to do in this PR, but this try/except ImportError technique to import these modules in different runtimes is starting to seem more and more cumbersome as we add more and more modules. 😞
We should file an issue to reorganize the modules in such a way that they can be imported in all the necessary runtimes AND we can stop using this try/except ImportError technique.
There was a problem hiding this comment.
Filed #148 to reorganize the modules and drop the shim.
Review follow-ups on #147: - Collapse the duplicated dismiss/not-dismiss main() tests into two parametrized tests driven by NOT_DISMISSED_CASES / DISMISSED_CASES. - Rename severity_l -> severity_lower for clarity.
What
Adds
investigate.dismiss_max_severity(low|medium|high|critical, default medium) — the highest severity BLEnder may auto-dismiss. Replaces the previously hardcoded high/critical block.Combined rule (with #145)
An alert is auto-dismissed only when:
not affectedANDseverity ≤ dismiss_max_severityANDconfidence ≥ dismiss_min_confidence.Confidence is the safety rail; severity is the opt-in scope. A repo confident it's unaffected can raise the ceiling all the way to
critical— but a critical still only dismisses at the confidence floor (defaulthigh).Details
dismiss_max_severitywarns and falls back tomedium.severity X above ceiling Y) is surfaced in the step summary.criticalceiling, critical blocked below ceiling (with summary note), invalid-value fallback.Closes #144.