Skip to content

Add weekly PR triage agentic workflow#34355

Open
jfversluis wants to merge 4 commits intomainfrom
feature/weekly-pr-triage
Open

Add weekly PR triage agentic workflow#34355
jfversluis wants to merge 4 commits intomainfrom
feature/weekly-pr-triage

Conversation

@jfversluis
Copy link
Member

Note

Are you waiting for the changes in this PR to be merged?
It would be very helpful if you could test the resulting artifacts from this PR and let us know in a comment if this change resolves your issue. Thank you!

Description

Adds a gh-aw (GitHub Agentic Workflows) workflow that runs weekly to scan all open PRs and surface "low-hanging fruit" — PRs that can be moved forward with minimal maintainer effort.

Why

The dotnet/maui repository typically has 50-100+ open PRs at any time. Many are actionable but get lost in the noise:

  • PRs that are approved + CI green but nobody clicked merge
  • Bot dependency bumps piling up and causing cascading conflicts
  • Community PRs with zero reviewer engagement after weeks
  • Duplicate PRs fixing the same issue
  • PRs fixing issues that were already resolved another way
  • Author addressed review feedback but the reviewer never came back

This workflow automates the detection of these patterns and produces a single weekly triage report as a GitHub issue.

What It Does

Uses the gh-aw agentic workflow framework (same as ci-doctor.md and daily-repo-status.md already in this repo) to iterate over every open PR and evaluate it against 20 criteria across 4 priority tiers:

Tier Focus Examples
🔴 Immediate Merge or close today Approved+green, bot deps, already-fixed issues, duplicates
🟠 Quick wins < 30 min effort Small community PRs, addressed feedback awaiting re-review, docs/test-only
🟡 Hygiene Clean up the queue Wrong branch, merge conflicts, stuck labels, abandoned PRs
🔵 Patterns Systemic insights Unrelated platform CI failures, partner blockers, overlapping PRs

Each weekly run creates a single GitHub issue with the report and auto-closes the previous week's issue (close-older-issues: true).

Design Decisions

  • gh-aw format — follows the exact pattern of existing ci-doctor.md and daily-repo-status.md. Run gh aw compile to generate the .lock.yml.
  • gpt-5.1-codex model — good balance of reasoning and cost for iterating over many PRs
  • 20 criteria from 3-model brainstorm — criteria were developed by consulting Claude Opus, GPT-5.1, and Gemini independently, then synthesizing the best ideas
  • Structured output — collapsible sections, tables, per-PR action items. Empty categories are skipped.
  • MAUI-specific context — understands branching policy (main vs net10.0), label taxonomy (p/0-p/4, partner, community), milestone conventions, auto-generated files, and bot authors

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings March 6, 2026 12:03
@github-actions
Copy link
Contributor

github-actions bot commented Mar 6, 2026

🚀 Dogfood this PR with:

⚠️ WARNING: Do not do this without first carefully reviewing the code of this PR to satisfy yourself it is safe.

curl -fsSL https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.sh | bash -s -- 34355

Or

  • Run remotely in PowerShell:
iex "& { $(irm https://raw.githubusercontent.com/dotnet/maui/main/eng/scripts/get-maui-pr.ps1) } 34355"

Run gh aw compile to generate weekly-pr-triage.lock.yml.
Fix issues: write -> issues: read (use safe-outputs instead).
Add actions: read for GitHub Actions toolset.
Other lock files regenerated by same compile step.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new GitHub Agentic Workflows (gh-aw) workflow specification to generate a weekly “low-hanging fruit” triage report across all open PRs in dotnet/maui, publishing results as a single GitHub issue.

Changes:

  • Introduces a weekly PR triage gh-aw workflow spec with 20 criteria grouped into 4 priority tiers.
  • Configures issue creation output (labels, title prefix, auto-close of older reports) for a single weekly report.
  • Defines a structured report format (collapsible sections + tables) and sorting rules for actionable items.

- Switch from gpt-5.1-codex to gpt-5.1-codex-mini to align with
  existing workflows and reduce cost
- Use authorAssociation (MEMBER/OWNER) instead of org membership
  check to avoid requiring read:org permissions

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@PureWeen
Copy link
Member

PureWeen commented Mar 6, 2026

Can we make sure this surfaces Milestoned PRs still?

I get the low hanging fruit direction but from a priority standpoint we really need to stay focused on Milestoned PRs.

I worry about too many signals slowing down high priority areas.

Addresses feedback from @PureWeen to ensure milestoned PRs are always
surfaced prominently. Adds a dedicated Tier 0 section that:
- Always appears first in the report, never collapsed
- Shows ALL milestoned PRs grouped by milestone (lowest SR first)
- Includes state, CI status, blocker, and idle days per PR
- Provides specific per-PR unblock recommendations

This ensures the team stays focused on milestoned work while still
surfacing low-hanging fruit opportunities below.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@jfversluis
Copy link
Member Author

Great point! Added a dedicated Tier 0 — Milestoned PR Dashboard that always appears first in the report (never collapsed, never skipped). It shows ALL milestoned PRs grouped by milestone (lowest SR first) with state, CI status, specific blockers, and idle days.

This ensures the milestoned work stays front-and-center while the lower tiers (quick wins, queue hygiene, pattern detection) are collapsed below for optional cleanup when time permits.

See commit d264cfd.

@MauiBot
Copy link
Collaborator

MauiBot commented Mar 23, 2026

🤖 AI Summary

📊 Expand Full Reviewd264cfd · Add Tier 0 milestoned PR dashboard as top priority
🔍 Pre-Flight — Context & Validation

Issue: None linked in PR description
PR: #34355 - Add weekly PR triage agentic workflow
Platforms Affected: GitHub Actions / repository automation (no app runtime platform)
Files Changed: 6 implementation/infrastructure, 0 test

Key Findings

  • PR adds a new gh-aw workflow spec (.github/workflows/weekly-pr-triage.md) plus generated workflow output (.github/workflows/weekly-pr-triage.lock.yml) and updates existing generated lock files plus .github/aw/actions-lock.json.
  • No test projects or UI/device/unit test files are changed; validation will depend on workflow/configuration checks rather than app tests.
  • Prior inline review feedback focused on missing compiled lock workflow and missing actions: read permission; those review threads are resolved in the current PR.
  • A maintainer requested that milestoned PRs remain prominently surfaced; author responded that Tier 0 now always shows a milestoned PR dashboard first.

Fix Candidates

# Source Approach Test Result Files Changed Notes
PR PR #34355 Add weekly gh-aw workflow/report plus generated lock workflow and supporting lockfile updates for weekly PR triage reporting ⏳ PENDING (Gate) .github/aw/actions-lock.json, .github/workflows/agentics-maintenance.yml, .github/workflows/ci-doctor.lock.yml, .github/workflows/daily-repo-status.lock.yml, .github/workflows/weekly-pr-triage.lock.yml, .github/workflows/weekly-pr-triage.md Original PR; no linked issue

🚦 Gate — Test Verification

Gate Result: ⚠️ SKIPPED

Platform: android
Mode: Full Verification

  • Tests FAIL without fix: N/A (no relevant tests in PR)
  • Tests PASS with fix: N/A (no relevant tests in PR)

Notes:

  • gh pr view 34355 --json files shows only workflow/lockfile changes and no UI/device/unit test files.
  • Per gate instructions, validation is skipped when no tests exist for the PR.

🔧 Fix — Analysis & Comparison

Fix Candidates

# Source Approach Test Result Files Changed Notes
1 try-fix (claude-opus-4.6) Consolidate weekly PR triage into existing daily-repo-status.md with a weekly gate instead of a standalone workflow ⚠️ BLOCKED 1 file gh aw CLI unavailable; baseline script excludes .github/* so PR files could not be auto-baselined
2 try-fix (claude-sonnet-4.6) Keep standalone workflow but add auto-sunset/issue-expiry lifecycle using stop-after and expires ⚠️ BLOCKED 1 file Also requires gh aw compile; baseline script still excludes .github/*
3 try-fix (gpt-5.3-codex) Centralize weekly scheduling in agentics-maintenance.yml and make triage dispatch-only ⚠️ BLOCKED 3 files Blocked by missing gh aw; also constrained by .github/* baseline exclusion
4 try-fix (gemini-3-pro-preview) Trigger triage via workflow_run from the daily repo status workflow instead of its own schedule ⚠️ BLOCKED 2 files Source edits possible, but no way to regenerate/validate lock output without gh aw
5 try-fix (claude-opus-4.6, round 2) Convert weekly triage into reusable workflow_call workflow invoked by maintenance infrastructure ⚠️ BLOCKED 4 files Different architecture, same blocker: no gh aw and no .github/* baseline support
6 try-fix (claude-sonnet-4.6, round 2) Use a heartbeat/day-of-week gate inside agentics-maintenance.yml instead of standalone triage workflow files ⚠️ BLOCKED 1 file Architecturally distinct, but mandated validation still blocked by missing gh aw
7 try-fix (gemini-3-pro-preview, round 2) Directly edit compiled weekly-pr-triage.lock.yml instead of regenerating from .md ⚠️ BLOCKED 1 file Avoids source regeneration but still leaves no supported validation path and diverges from generated-file workflow
PR PR #34355 Add standalone weekly gh-aw triage workflow spec, compiled lock workflow, and lockfile updates ⚠️ SKIPPED (Gate) 6 files Original PR; no relevant tests in PR

Cross-Pollination

Model Round New Ideas? Details
claude-opus-4.6 2 Yes Reusable workflow_call workflow invoked from maintenance schedule
claude-sonnet-4.6 2 Yes Heartbeat pattern: existing scheduled workflow checks day-of-week before running triage
gpt-5.3-codex 2 No No materially new ideas beyond blocked architecture variants
gemini-3-pro-preview 2 Yes Direct manual edits to compiled .lock.yml instead of regenerating from .md
claude-opus-4.6 3 Yes External scheduler triggers triage via repository_dispatch
claude-sonnet-4.6 3 No No further in-repo ideas beyond already-tried variants
gpt-5.3-codex 3 Yes Azure DevOps scheduled pipeline instead of GitHub workflow
gemini-3-pro-preview 3 Yes Azure DevOps pipeline variant of external scheduler

Exhausted: Yes (max 3 cross-pollination rounds reached)
Selected Fix: PR #34355 — no empirically validated alternative outperformed the PR, and round-3 ideas drifted out of scope into external scheduling/Azure DevOps rewrites rather than same-shape workflow improvements.


📋 Report — Final Recommendation

⚠️ Final Recommendation: REQUEST CHANGES

Phase Status

Phase Status Notes
Pre-Flight ✅ COMPLETE Workflow-only PR; no linked issue; prior review feedback resolved
Gate ⚠️ SKIPPED No UI/device/unit tests or other relevant test assets were changed in the PR
Try-Fix ✅ COMPLETE 7 attempts, 0 passing; all blocked by missing gh aw and .github/* baseline exclusions
Report ✅ COMPLETE

Summary

The PR is close, and the overall gh-aw workflow design is reasonable. However, it introduces one substantive workflow-security/regeneration issue: agentics-maintenance.yml uses github/gh-aw/actions/setup-cli@v0.53.5 by tag, but .github/aw/actions-lock.json only registers SHA-pinned entries for github/gh-aw/actions/setup@..., not setup-cli. That leaves the new action outside the repository's existing action-locking pattern.

Root Cause

The generated maintenance workflow added a new gh-aw action (setup-cli) without a corresponding locked entry in .github/aw/actions-lock.json, and the workflow reference itself is tag-pinned rather than SHA-pinned. In a repo that already tracks gh-aw action pins centrally, this is a correctness/security gap: the new dependency is not covered by the same lock mechanism as the rest of the gh-aw actions.

Fix Quality

Aside from that issue, the PR addresses the earlier review feedback well: it includes the compiled lock workflow, adds actions: read, and keeps the weekly triage workflow aligned with existing gh-aw patterns. I did not find a better validated alternative in try-fix; all alternative architectures were blocked by missing local gh aw tooling. My recommendation is to keep the current design, but fix the new setup-cli action reference so it is locked/pinned consistently with the rest of the repo.


@MauiBot MauiBot added s/agent-changes-requested AI agent recommends changes - found a better alternative or issues s/agent-fix-pr-picked AI could not beat the PR fix - PR is the best among all candidates s/agent-reviewed PR was reviewed by AI agent workflow (full 4-phase review) labels Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

s/agent-changes-requested AI agent recommends changes - found a better alternative or issues s/agent-fix-pr-picked AI could not beat the PR fix - PR is the best among all candidates s/agent-reviewed PR was reviewed by AI agent workflow (full 4-phase review)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants