Frontend triage severity as comment suggestion - #6679
Merged
crythms merged 4 commits intoAug 20, 2026
Merged
Conversation
The agent wrote the severity it judged straight to the bug's `severity` field, and at `confidence: high` that landed with nobody in between. Keep the judgment, but deliver it as a suggestion a human applies: a two-line block at the end of the triage comment, carrying `(currently S4)` when it disagrees with what is on the bug. `severity` was the only field any ruleset directed the agent to change -- `config.py` noted that no rule asks for a keyword -- so moving it left `bugzilla.update_bug` with no caller. Dropping it from ENABLED_ACTION_TYPES takes the field-change hook, its two validators and TRIAGE_FIELDS / TRIAGE_KEYWORDS with it, and removes the `editbugs` requirement on the apply account: the apply step coalesces a same-bug field change with the nearest comment into one PUT, so a rejected change used to take the analysis comment down with it. A comment is now the only thing the agent can write. The block is withheld when the agent is not confident in the impact judgment. That is `severity_assessment.confidence`, not the run's top-level confidence about localizing the cause -- the two are independent, and the rules conflated them. `notify.py` gates its S1 marker on the same threshold so Slack cannot shout S1 at a team while the bug says nothing about severity, and names it `(suggested S1)` now that nothing is written. Both fields are normalized in `parse_plan` first. `severity_assessment` was built straight from the model's free-form JSON, so a model writing "High" would have read as "not high" and silently dropped both the comment block and the Slack marker. TRIAGE_SEVERITIES survives to validate the level.
The old prompt asked the model to "state the severity conclusion" in the comment with no format, so it landed wherever the model chose. That instruction is gone, replaced by the block at the end -- but nothing stops the model from also writing the level into its analysis, leaving a reader two declarations that can disagree. Guards the declaration, not the mention: the regex is anchored to the line start, so quoting a reporter or arguing why something is not S1 does not count. Absence still passes, since a run with low severity confidence omits the block on purpose.
Drop the `(currently S4)` parenthetical. A reader has the severity field on screen, so it restated something visible -- and worse, a comment is permanent while the field is not, so "currently S4" is wrong the moment anyone changes it. `(currently unset)` was redundant for the same reason. This also removes the only part of the block that required reading and comparing the bug's current severity. `Severity assessment:` becomes `Suggested severity:`, which answers "did the bot change something?" in the label itself, and the `Rationale:` prefix goes -- the sentences sit under the level.
No blank line between them. BMO renders every comment through `markdown-body` with hard line breaks -- otherwise two decades of hard-wrapped plain-text comments would reflow -- so a single newline is a line break, not a paragraph join. The blank line before the `---` stays: without it the preceding line parses as a setext heading.
msujaws
approved these changes
Aug 20, 2026
Contributor
|
Thanks! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The agent used to write severity straight to the bug's
severityfield. Atconfidence: highthat happened with no human in between. It still works out a severity — it just suggests it at the end of the comment now, before the footer:The block is left out entirely when the agent isn't confident about the impact. A level it's unsure of still reads like a judgement someone might act on so we avoid that.
Other relevant changes
Which confidence decides whether the block sev appears. The agent reports two, and they measure different things:
confidence— how sure it is that it found the cause in the codeseverity_assessment.confidence— how sure it is about the user impactThe block is gated on the second one. They're independent: the agent can be certain a bug is cosmetic while having no idea which file is at fault, and that bug should still get a severity suggestion. The rules didn't distinguish the two before; they
do now.
notify.pyuses the same threshold for its Slack S1 marker, so Slack can't flag an S1 on a bug whose comment says nothing about severity.severity_assessmentis now normalized inparse_plan. It came straight from the model's JSON before, so"High"instead of"high"would have silently dropped both the comment block and the Slack marker.