Workflow Health Dashboard — 2026-03-11 #20479
Description
This issue tracks the operational health of all 166 agentic workflows in this repository as of 2026-03-11.
Summary
| Category | Count | % |
|---|---|---|
| ✅ Healthy | 158 | 95% |
| 4 | 2% | |
| ❌ Critical | 4 | 2% |
| 🔇 Inactive | 0 | 0% |
Overall Health Score: 72/100 (↑2 from 70 — Smoke Codex and Duplicate Code Detector recovered)
Critical Issues 🚨
P1: Lockdown Token Missing — 4 Workflows (Day 15+)
All four workflows fail consistently due to lockdown: true requiring GH_AW_GITHUB_TOKEN which is not configured as a repository secret.
| Workflow | Frequency | Run # | Error |
|---|---|---|---|
| Issue Monster | Every 30 min | #2692 | Lockdown token missing |
| PR Triage Agent | Every 6h | #192 | Lockdown token missing |
| Daily Issues Report | Daily | #129 | Lockdown token missing |
| Org Health Report | Weekly | #28 | Lockdown token missing |
- Action: Tracking issue #20315 — OPEN
- Root cause:
GH_AW_GITHUB_TOKENsecret not provisioned; all programmatic fix paths closed - Error:
Lockdown mode is enabled (lockdown: true) but no custom GitHub token is configured - Priority: P1 — requires manual admin intervention to provision secret
Warnings ⚠️
P2 Issues with Tracking (3 workflows)
| Workflow | Issue | Error | Last Run |
|---|---|---|---|
| Safe Output Health Monitor | #20305 | Agent job failure (2 consecutive) | Mar 11 #160 |
| Smoke Update Cross-Repo PR | #20288 | Pre-agent failure | Mar 11 #118 |
| Smoke Codex | #20285 |
Codex engine issue (was OpenAI restriction) | SUCCESS Mar 11 #2215 |
P2 Issues without Tracking (2 workflows)
- Smoke Gemini — 100% failure rate on schedule (run [Custom Engine Test] Test Issue Created by Custom Engine #322 Mar 11) — likely
add_commentcontext error on schedule trigger - jsweep — Intermittent failure (1/10 recent runs) — agent ran but produced no output, orphan processes terminated (run MCP Network Permissions Test Results - Proxy Isolation Analysis #109 Mar 11)
Recoveries 🎉
| Workflow | Previous State | Current State | Run |
|---|---|---|---|
| Smoke Codex | ❌ Failing 2+ weeks (OpenAI restriction) | ✅ RECOVERED | #2215 Mar 11 |
| Duplicate Code Detector | ❌ Failing Mar 7–10 | ✅ RECOVERED | #230 Mar 11 |
Both Codex-engine workflows are now back to healthy. Open tracking issues #20285 and #20304 can be closed.
Compilation Status ✅
- 166/166 workflows compiled successfully
- 0 missing lock files
- 0 genuinely stale lock files (false positives previously reported are git checkout timestamp artifacts)
Systemic Issues
GH_AW_GITHUB_TOKEN Missing (P1 — Ongoing Day 15+)
- Affected: Issue Monster, PR Triage Agent, Daily Issues Report, Org Health Report
- Pattern: All use
lockdown: truerequiring a custom GitHub token - Status: All programmatic fix paths closed — requires admin action
- Impact: Issue tracking (~50+ failed runs/day), PR triage, and daily/weekly reporting degraded
Healthy Workflows ✅
158 workflows operating normally, including:
- Smoke Copilot ✅ | Smoke Codex ✅ (RECOVERED) | Smoke Claude ✅ | Smoke Gemini (scheduled)
⚠️ - Metrics Collector ✅ | Agentic Maintenance ✅ | Chroma Issue Indexer ✅
- Auto-Triage Issues ✅ | Bot Detection ✅ | Duplicate Code Detector ✅ (RECOVERED)
- Contribution Check ✅ | Static Analysis Report ✅ | AI Moderator ✅ (mostly healthy)
Trends (7-Day)
| Date | Score | P1/P2 Failures | Notes |
|---|---|---|---|
| Mar 7 | 74/100 | 6 P1+P2 | Codex issues begin |
| Mar 9 | 72/100 | 8+ | Multiple new P2 failures |
| Mar 10 | 70/100 | 10+ | Codex + lockdown + smoke failures |
| Mar 11 | 72/100 | 6 | ↑ Codex + Duplicate Code recovered |
Recommendations
- URGENT: Provision
GH_AW_GITHUB_TOKENsecret — 4 workflows blocked daily ([P1] Lockdown token failures: Issue Monster, PR Triage Agent, Daily Issues Report #20315) - HIGH: Close [aw] Smoke Codex failed #20285 and [aw] Duplicate Code Detector failed #20304 — Smoke Codex and Duplicate Code Detector have recovered
- MEDIUM: Investigate Smoke Gemini
add_commentfailure on schedule trigger (needs issue context guard) - MEDIUM: Investigate Smoke Update Cross-Repo PR ongoing failure ([aw] Smoke Update Cross-Repo PR failed #20288)
- LOW: Monitor jsweep intermittent failures — may self-correct
Last updated: 2026-03-11T07:29:00Z
Run: §22941596501
Next check: 2026-03-12T07:00Z
Related to #19352
Generated by Workflow Health Manager - Meta-Orchestrator · ◷
- expires on Mar 12, 2026, 7:42 AM UTC