Add severity assessment to frontend-triage agent

agents/frontend-triage/hackbot_agents/frontend_triage/agent.py

@@ -40,6 +40,7 @@
     permalink_prefix,
     resolve_index_revision,
 )
+from pydantic import BaseModel, ValidationError
 from searchfox import AsyncSearchfoxClient
 
 from .config import (
@@ -73,6 +74,14 @@ def feedback_tags_hook(action: dict) -> None:
         params["text"] = f"{text.rstrip()}\n{_FEEDBACK_TAGS}"
 
 
+class SeverityAssessment(BaseModel):
+    """Severity judgment (see severity-assessment rules)."""
+
+    suggested: str | None = None  # S1 | S2 | S3 | S4
+    confidence: str | None = None  # high | medium | low
+    rationale: str | None = None
+
+
 class FrontendTriageResult(HackbotAgentResult):
     bug_id: int
     # Structured plan (best-effort, parsed from the agent's final message).
@@ -87,6 +96,8 @@ class FrontendTriageResult(HackbotAgentResult):
     relevant_tests: list[str] | None = (
         None  # existing tests covering the area (verify anchor)
     )
+    # Triage judgments (best-effort, parsed from the agent's final message).
+    severity_assessment: SeverityAssessment | None = None
     # The agent's full final message, always present as a fallback.
     result: str | None = None
 
@@ -180,6 +191,14 @@ def _as_list(value):
             return [value]
         return value if isinstance(value, list) else None
 
+    def _as_model(model, value):
+        if not isinstance(value, dict):
+            return None
+        try:
+            return model.model_validate(value)
+        except ValidationError:
+            return None
+
     actionable = data.get("actionable")
     if not isinstance(actionable, bool):
         actionable = None
@@ -192,6 +211,9 @@ def _as_list(value):
         "actionable": actionable,
         "regressor_node": data.get("regressor_node"),
         "relevant_tests": _as_list(data.get("relevant_tests")),
+        "severity_assessment": _as_model(
+            SeverityAssessment, data.get("severity_assessment")
+        ),
     }
 
 

agents/frontend-triage/hackbot_agents/frontend_triage/prompts/system.md

@@ -8,7 +8,8 @@ You are given a bug ID. Your job is to triage it and produce a **proposed fix pl
 2. **Read the relevant triage rules** from `{rules_dir}` — Glob the directory and Read only the rulesets that apply to this bug. Do not assume all rules apply to all bugs.
 3. **Assess** what the rules say should happen, and whether the bug has open questions in its comments.
 4. **Investigate** the source tree (read-only) to localize the cause — delegate deep searches to the `investigator` subagent (see below).
-5. **Produce a fix plan**: the likely root cause, the specific files to change, and the approach. Record it as a brief Bugzilla comment.
+5. **Assess severity** — determine an appropriate Mozilla severity (S1–S4) from the user impact (see the `severity-assessment` rules).
+6. **Produce a fix plan**: the likely root cause, the specific files to change, and the approach. Record it as a brief Bugzilla comment.
 
 # This agent is READ-ONLY
 
@@ -86,7 +87,7 @@ Before calling any action tool, state in your response:
 - **What** action you are recording and **why** (cite the specific rule)
 - **Your confidence**: high / medium / low
 
-Record exactly one `bugzilla_add_comment` with your fix plan. Only record a `bugzilla_update_bug` (e.g. keyword/severity) when confidence is **high** and a specific triage rule directs it. Never record `status: RESOLVED`.
+Record exactly one `bugzilla_add_comment` with your fix plan (which should also state the severity conclusion). Only record a `bugzilla_update_bug` when confidence is **high** and a specific triage rule directs it — e.g. a `severity` (per the `severity-assessment` rules) or an obvious keyword. You may combine several such fields into one `bugzilla_update_bug`, each justified in the `reasoning`. At medium/low confidence, state the assessment in the comment and structured output but do **not** record a field change. Never record `status: RESOLVED`.
 
 The `reasoning` parameter on every action tool is required and stored alongside the recorded action. Fill it properly.
 
@@ -105,7 +106,12 @@ After recording your comment, end your final message with a fenced ```json block
   "confidence": "high | medium | low",
   "actionable": true,
   "regressor_node": "hg node of the introducing changeset, or null",
-  "relevant_tests": ["browser/.../tests/browser/browser_foo.js"]
+  "relevant_tests": ["browser/.../tests/browser/browser_foo.js"],
+  "severity_assessment": {{
+    "suggested": "S1 | S2 | S3 | S4",
+    "confidence": "high | medium | low",
+    "rationale": "user-impact reasoning"
+  }}
 }}
 ```
 
@@ -114,6 +120,7 @@ Field guidance for the handoff:
 - **`actionable`** — `false` when the bug is out of scope or skipped per the scoping rules (meta/tracking, intermittent/test-infra, enhancement/task), or when there is simply nothing to fix-plan; `true` when you produced a real fix plan. The executor uses this to decide whether to act.
 - **`regressor_node`** — when the bug is a regression and you identified/confirmed the introducing changeset (via the `mozilla_vcs` tools or `get_blame`), put its hg node here so the executor has a direct pointer; otherwise `null`.
 - **`relevant_tests`** — existing tests that cover the affected area (typically browser-chrome mochitests under a component's `tests/browser/` dir, or xpcshell tests). These are the executor's **verification anchor** — it can run them. Use `[]` if you searched and found none (a signal that the executor should add a test).
+- **`severity_assessment`** — the severity you judged appropriate (per the `severity-assessment` rules), with `confidence` and a `rationale`. Set to null only if you could not assess it.
 
 If you could not localize a root cause, set `root_cause` to null, keep `confidence` low, set `actionable` accordingly, and have your comment ask the specific open questions that block triage.
 

agents/frontend-triage/hackbot_agents/frontend_triage/rules/frontend-triage.md

@@ -25,6 +25,9 @@ ruleset does not apply — note that and stop.
 3. **Write a fix plan**: root cause, the specific files/functions/selectors to
    change, and the approach. Prefer a comprehensive fix at the right level over a
    spot fix.
+4. **Assess severity.** Apply the `severity-assessment` rules to judge the bug's
+   severity from its user impact. It belongs in your comment and structured
+   output.
 
 ## Comment
 
@@ -38,8 +41,9 @@ not restate the whole bug. Do not claim the fix is verified — you did not run
 
 - **High** (you found the specific code and the cause is clear): record the
   plan comment. If a rule or convention clearly applies, you may also record a
-  `bugzilla_update_bug` for an obviously-correct field (e.g. adding a relevant
-  keyword). Do not change `status`/`resolution`.
+  `bugzilla_update_bug` for an obviously-correct field — e.g. adding a relevant
+  keyword, or a `severity` (per `severity-assessment`). Do not change
+  `status`/`resolution`.
 - **Medium** (plausible area, cause not pinned down): record the comment with
   your best hypothesis and the open questions that would confirm it.
 - **Low** (could not localize): record a comment stating what you checked and

agents/frontend-triage/hackbot_agents/frontend_triage/rules/severity-assessment.md

@@ -0,0 +1,36 @@
+# Severity assessment
+
+Assess an appropriate Mozilla severity for the bug and record it in the
+`severity_assessment` structured-output object. Base the judgment on **user impact and
+reach** as evidenced by the bug report and the code you investigated — how badly the
+user is affected, how many users hit it, and whether a workaround exists.
+
+## Severity definitions
+
+- **S1 — catastrophic.** Crash, hang, data loss, security issue, or a bug that blocks
+  major functionality with **no workaround**. Affects a large number of users.
+- **S2 — serious.** Major functionality is broken or a severe UX problem, and the
+  workaround (if any) is painful or non-obvious. Affects many users.
+- **S3 — normal.** Blocks non-critical functionality, or a reasonable workaround exists.
+  **This is the default for most frontend papercuts.**
+- **S4 — minor / trivial.** Cosmetic issues, small polish, or edge cases with negligible
+  impact.
+
+## Guidance
+
+- Frontend UI/UX papercuts are usually **S3** (or **S4** when purely cosmetic). Reserve
+  **S1 / S2** for genuine breakage: crashes, data/state loss, or a broken core workflow
+  with no easy workaround.
+- Weigh: is it functional vs cosmetic? Is there a workaround? How frequently and how
+  broadly is it hit (mainline path vs rare configuration)?
+- Do **not downgrade** an existing higher severity unless you have strong evidence the
+  impact is lower than currently recorded.
+
+## Confidence and field changes
+
+- **High** — impact is clear-cut (clearly cosmetic, or clearly a crash/data-loss). Only
+  then may you record a `bugzilla_update_bug` proposing the `severity` (see the system
+  prompt's recording rules), with a `reasoning` citing the impact evidence. Prefer not to
+  propose a change when the bug already carries a reasonable severity.
+- **Medium / low** — suggest a severity in the comment and structured output, but do
+  **not** record a field change.