diff --git a/workflows/acs-triage/.claude/commands/comment-issues.md b/workflows/acs-triage/.claude/commands/comment-issues.md index a8bdc3e..ce8553c 100644 --- a/workflows/acs-triage/.claude/commands/comment-issues.md +++ b/workflows/acs-triage/.claude/commands/comment-issues.md @@ -56,13 +56,12 @@ Add triage analysis comments to JIRA issues based on team assignment results. Po - Total issues processed - Successfully commented - Failures (with reasons) - - Skipped (low confidence <70%) ## Safety Guardrails -**Confidence Filter:** -- Only comment on issues with confidence ≥70% -- Flag low-confidence issues for manual review +**No Confidence Filtering:** +- Comments ALL triaged issues regardless of confidence level +- Low-confidence issues (<80%) include ⚠️ warning: "Low confidence - requires manual review" **Dry Run Mode:** - Default: dry_run = true diff --git a/workflows/acs-triage/.claude/commands/triage.md b/workflows/acs-triage/.claude/commands/triage.md index 02ac708..85877b2 100644 --- a/workflows/acs-triage/.claude/commands/triage.md +++ b/workflows/acs-triage/.claude/commands/triage.md @@ -6,9 +6,9 @@ Complete end-to-end triage workflow for StackRox/ACS JIRA issues. Fetches untria **Options:** - `/triage` - Full triage pipeline using JQL search (READ-ONLY, no JIRA writes) -- `/triage --comment` - Full triage + post comments to JIRA + add auto-triaged label -- `/triage ROX-12345` - Triage a specific issue by key -- `/triage ROX-12345 --comment` - Triage specific issue, post comment, and add label +- `/triage --comment` - Full triage + post comments to ALL issues (regardless of confidence, <80% get ⚠️ warning) + add auto-triaged label +- `/triage ROX-12345` - Triage a specific issue by key (READ-ONLY) +- `/triage ROX-12345 --comment` - Triage specific issue + post comment + add label ## Prerequisites @@ -26,7 +26,7 @@ This command executes the following phases: **PERFORMANCE OPTIMIZATION:** These phases have no interdependencies and SHOULD run concurrently to save 10-20 seconds. #### Phase 1a: Setup (if needed) - Async -Clone StackRox repository for CODEOWNERS and reference data if not already present. +Clone required repositories for CODEOWNERS, reference data, and skills if not already present. **Actions:** - Check if `/tmp/triage/stackrox/.github/CODEOWNERS` exists @@ -35,6 +35,11 @@ Clone StackRox repository for CODEOWNERS and reference data if not already prese - Check if `/tmp/triage/stackrox/.claude/agents/stackrox-ci-failure-investigator.md` exists - If present: deep CI failure analysis will use this agent's methodology - If missing: log warning "CI failure investigator agent not found - deep analysis will use description-only mode" +- Check if `/tmp/triage/skills/.claude/skills/` exists +- If missing, clone `https://github.com/stackrox/skills` to `/tmp/triage/skills` +- Skills available for use: + - `/tmp/triage/skills/.claude/skills/rhacs-patch-eval/` - Patch evaluation and VEX analysis + - Any other skills in the repository can be loaded on-demand **Output:** Setup metadata in `artifacts/acs-triage/setup-info.json` @@ -249,10 +254,10 @@ Create output reports in two formats. ``` ### Phase 7: Comment to JIRA and Add Label (Optional) -Only if `--comment` flag is provided. +Only executed if `--comment` flag is provided. When enabled, processes ALL triaged issues regardless of confidence level. **Actions:** -- For each issue with confidence ≥80%: +- For each triaged issue (NO CONFIDENCE FILTERING): - **Convert team mention:** Convert GitHub team handle to JIRA team mention - Use mapping from `reference/jira-team-mappings.md` - Format: `[Team Display Name](https://redhat.atlassian.net/jira/people/team/{team-id}?ref=jira$&src=issue)` @@ -261,13 +266,14 @@ Only if `--comment` flag is provided. - **Post comment:** Post structured comment with team recommendation, confidence, reasoning - Use comment format from `templates/jira-comment.md` - Use `mcp__mcp-atlassian__jira_add_comment` + - Include confidence level in comment (always show percentage) + - For low confidence (<80%), add warning: "⚠️ Low confidence - requires manual review" - **Add label:** After successful comment post, add `auto-triaged` label to the issue - Use `mcp__mcp-atlassian__jira_update_issue` with `labels` parameter - Append to existing labels (don't replace) - Example: `{"labels": ["auto-triaged"]}` - **Log:** Record issue key, team, confidence, comment status, and label status -- Skip issues with low confidence (<80%) -- Log all posted comments, labels added, and skipped issues (with reason) +- Log all posted comments and labels added (no issues are skipped based on confidence) **Comment Template:** See `templates/jira-comment.md` for format and variable substitution. @@ -290,13 +296,23 @@ Triage all untriaged issues (read-only): ``` /triage ``` +This will: +- Fetch all untriaged issues (without `auto-triaged` label) +- Classify, analyze, and assign teams +- Generate reports (no JIRA writes) Triage all untriaged issues and post comments: ``` /triage --comment ``` - -Triage a specific issue: +This will: +- Fetch all untriaged issues +- Classify, analyze, and assign teams +- Post triage comments to ALL issues (regardless of confidence level) +- Low confidence (<80%) issues get ⚠️ warning in comment +- Add `auto-triaged` label to prevent duplicate processing + +Triage a specific issue (read-only): ``` /triage ROX-12345 ``` diff --git a/workflows/acs-triage/CLAUDE.md b/workflows/acs-triage/CLAUDE.md index 76e881f..bbbf338 100644 --- a/workflows/acs-triage/CLAUDE.md +++ b/workflows/acs-triage/CLAUDE.md @@ -19,9 +19,11 @@ This is a **single-purpose workflow** for automated triage of StackRox/ACS JIRA The workflow provides 2 main commands: - `/triage` - Complete end-to-end triage pipeline: setup → fetch → classify → analyze → assign → report (READ-ONLY) -- `/triage --comment` - Full triage pipeline + post analysis comments to JIRA + add auto-triaged label (⚠️ WRITES to JIRA) +- `/triage --comment` - Full triage pipeline + post analysis comments to ALL issues (regardless of confidence) + add auto-triaged label (⚠️ WRITES to JIRA) - `/comment-issues` - Standalone command to add triage comments to JIRA (requires prior /triage run) +**Note:** When using `--comment`, ALL triaged issues receive comments regardless of confidence level. Low confidence (<80%) issues include a ⚠️ warning flag in the comment for manual review. + **Simplified Design:** All triage steps are consolidated into a single `/triage` command for ease of use. **Idempotent Execution:** The workflow uses JQL search with `labels NOT IN (auto-triaged)` exclusion. After triaging an issue and posting a comment, the `auto-triaged` label is added. This makes the workflow safe to run repeatedly - only new untriaged issues will be processed. diff --git a/workflows/acs-triage/FIELD_REFERENCE.md b/workflows/acs-triage/FIELD_REFERENCE.md index 1b2e6d7..797e2f4 100644 --- a/workflows/acs-triage/FIELD_REFERENCE.md +++ b/workflows/acs-triage/FIELD_REFERENCE.md @@ -299,14 +299,7 @@ These fields are added by the `/analyze-vuln` command for VULNERABILITY issues: - **Example:** "github.com/stackrox/rox/scanner/pkg", "react-dom", "golang.org/x/net" - **Purpose:** Specific package/library affected with full import path - **Extracted From:** Issue description or CVE details -- **Note:** Used for Go dependency team assignment - -#### vuln_analysis.component -- **Type:** string -- **Example:** "scanner", "central", "ui" -- **Purpose:** Affected component/module (informational, secondary to language/package) -- **Extracted From:** Issue description or labels -- **Deprecated:** Use language + package_name for team assignment instead +- **Note:** Primary field for Go dependency team assignment (with language) #### vuln_analysis.decision_tree - **Type:** object diff --git a/workflows/acs-triage/README.md b/workflows/acs-triage/README.md index 84c7b3b..e65d32b 100644 --- a/workflows/acs-triage/README.md +++ b/workflows/acs-triage/README.md @@ -14,15 +14,13 @@ This workflow provides systematic triage of untriaged StackRox issues using: ### Workflow Phases -1. **Setup** - Clone StackRox repository for CODEOWNERS and reference data -2. **Fetch Issues** - Retrieve untriaged issues from JIRA filters (103399, 95004) +1. **Setup** - Clone StackRox repository for CODEOWNERS and stackrox/skills for reusable analysis skills +2. **Fetch Issues** - Retrieve untriaged issues from JIRA using JQL search 3. **Classify** - Categorize as CI_FAILURE, VULNERABILITY, FLAKY_TEST, or UNKNOWN 4. **Analyze** - Apply specialized analysis based on type 5. **Assign Team** - Multi-strategy assignment with confidence scoring -6. **Generate Reports** - Create markdown, HTML, and Slack outputs -7. **Review** - Human review of recommendations -8. **Comment** (Optional) - Add triage comments to JIRA -9. **Execute** - Manual JIRA updates based on report +6. **Generate Reports** - Create markdown and JSON outputs +7. **Comment** (Optional with --comment) - Add triage comments to ALL issues (regardless of confidence) + auto-triaged label ## Getting Started diff --git a/workflows/acs-triage/reference/constants.md b/workflows/acs-triage/reference/constants.md index fc60210..554e65e 100644 --- a/workflows/acs-triage/reference/constants.md +++ b/workflows/acs-triage/reference/constants.md @@ -25,10 +25,13 @@ Central location for all hardcoded values used throughout the ACS triage workflo | Range | Classification | Recommendation | |-------|---------------|----------------| | ≥90% | High | Ready for automatic assignment | -| 70-89% | Medium | Review before assignment | +| 80-89% | Medium-High | Safe for assignment with comment | +| 70-79% | Medium | Review before assignment | | <70% | Low | Manual review required | | 0% | None | Needs manual assignment | +**Low Confidence Warning Threshold**: 80% - Issues below this threshold receive "⚠️ Low confidence - requires manual review" warning in JIRA comments. + ## Severity Thresholds (Vulnerabilities) | Severity | CVSS Range | Triage Decision | @@ -113,7 +116,9 @@ Central location for all hardcoded values used throughout the ACS triage workflo ## Repository Paths -| Repository | Clone Path | Files Needed | -|-----------|-----------|--------------| -| stackrox/stackrox | /tmp/triage/stackrox | .github/CODEOWNERS, VERSION | -| stackrox/skills | /tmp/triage/skills | plugins/rhacs-patch-eval/* | +| Repository | Clone Path | Resources Needed | +|-----------|-----------|-----------------| +| stackrox/stackrox | /tmp/triage/stackrox | .github/CODEOWNERS, VERSION, .claude/agents/* | +| stackrox/skills | /tmp/triage/skills | .claude/skills/* (rhacs-patch-eval, etc.) | + +**Skills Repository:** Contains reusable skills for ACS-specific analysis tasks. Skills can be loaded on-demand during triage workflow execution.