[detection-analysis] Detection Analysis Report — 2026-07-04 (last 24h)

[github-actions[bot]]

Detection Analysis Report — Last 24h

Summary

• Window evaluated: last 24 full hours (UTC) — 2026-07-04T08:14:25Z → 2026-07-04T12:48:38Z
• Total runs analyzed: 100
• Detection-enabled runs (features.gh-aw-detection: true): 21 (21%)
• Regular runs (flag false/absent): 79 (79%)
• Misconfigured workflows found: 2

Warning

2 workflows are flagged as misconfigured this window. See the Misconfigured Workflows section for details and recommended fixes.

Metric	Regular Runs	Detection Runs
Total runs	79	21
Success rate	82.3%	81.0%
Failure count	14	4
Avg cost (AIC)1	~63	~84
Misconfigured	—	2

1 Token note: TokenUsage/EffectiveTokens were 0 across all 100 runs in this window's run_summary.json (token accounting not populated), so AIC (AI-cost units) from aw_info/summary is used as the usage proxy. Treat absolute values as directional.

Comparison Chart

Detection-enabled and regular runs performed almost identically on success rate (81.0% vs 82.3%) — detection is not degrading run reliability. Detection adoption sits at 21% of all runs, spread across copilot (13), claude (3), pi (3), and codex (2) engines.

Misconfigured Workflows

Workflow	Misconfiguration type	Runs	Recommended fix
Copilot Agent Prompt Clustering Analysis	Analysis workflow missing gh-aw-detection: true (name contains "Analysis")	1	Add gh-aw-detection: true — analysis/monitoring workflows should participate in detection. Example run §28703236869.
Smoke CI	gh-aw-detection: false on an active workflow (7 runs in window)	7	Likely intentional — this is a plain CI workflow (0 AIC, non-agentic, all 7 failing on unrelated CI errors). If confirmed non-agentic, add a suppression/allowlist entry so it stops matching the detection rule; otherwise enable detection. Example run §28704356594.

Not flagged as misconfig: The 4 detection-enabled failures (Daily Max Ai Credits Test, Changeset Generator, daily-experiment-report, Daily Rendering Scripts Verifier) failed on ordinary workflow errors — no detection-job-specific error patterns were found, so they are not counted as detection misconfigurations. No workflows showed inconsistent (mixed) detection state within the window.

View All Run Metrics (per-workflow breakdown)

Legend: Det = ✅ detection-enabled this window, — = regular. AIC = AI-cost proxy.

Workflow	Det	Runs	Success	AIC
AI Moderator	✅	1	1/1 (100%)	0
Agent Container Smoke Test	—	1	1/1 (100%)	13
Auto-Triage Issues	✅	1	1/1 (100%)	5
Changeset Generator	✅	1	0/1 (0%)	3
Claude Code User Documentation Review	✅	1	1/1 (100%)	425
Constraint Solving — Problem of the Day	✅	1	1/1 (100%)	7
Contribution Check	✅	1	1/1 (100%)	166
Copilot Agent Prompt Clustering Analysis	—	1	1/1 (100%)	157
Daily Documentation Updater	✅	1	1/1 (100%)	20
Daily Go Function Namer	✅	1	1/1 (100%)	9
Daily Max Ai Credits Test	✅	1	0/1 (0%)	7
Daily Rendering Scripts Verifier	✅	1	0/1 (0%)	0
Daily Syntax Error Quality Check	—	1	1/1 (100%)	19
Design Decision Gate 🏗️	—	9	9/9 (100%)	208
Dev	—	1	1/1 (100%)	11
Draft PR Cleanup	—	1	1/1 (100%)	28
ESLint Miner	—	1	1/1 (100%)	327
GitHub API Consumption Report Agent	✅	1	1/1 (100%)	232
Impeccable Skills Reviewer	—	9	8/9 (89%)	543
Instructions Janitor	—	1	1/1 (100%)	158
Issue Monster	—	5	5/5 (100%)	8
Matt Pocock Skills Reviewer	—	9	9/9 (100%)	717
PR Code Quality Reviewer	—	9	4/9 (44%)	1293
PR Description Updater	—	2	2/2 (100%)	70
PR Sous Chef	—	7	7/7 (100%)	61
PR Triage Agent	—	1	1/1 (100%)	56
Package Specification Enforcer	—	1	1/1 (100%)	19
Package Specification Extractor	—	1	1/1 (100%)	235
Smoke Antigravity	—	1	1/1 (100%)	0
Smoke CI	—	7	0/7 (0%)	0
Smoke Claude	—	1	1/1 (100%)	99
Smoke Codex	—	1	0/1 (0%)	7
Smoke Copilot	—	1	1/1 (100%)	143
Smoke Copilot - AOAI (Entra)	—	1	1/1 (100%)	12
Smoke Copilot - AOAI (apikey)	—	1	1/1 (100%)	10
Smoke Gemini	—	1	1/1 (100%)	0
Smoke Pi	—	1	1/1 (100%)	3
Stale PR Cleanup	—	1	1/1 (100%)	29
Team Status	—	1	1/1 (100%)	40
Terminal Stylist	—	1	1/1 (100%)	96
Test Quality Sentinel	✅	9	9/9 (100%)	358
Weekly Editors Health Check	—	1	1/1 (100%)	56
daily-experiment-report	✅	1	0/1 (0%)	447

View Historical Trend (last 30 days)

Run counts (both regular and detection) and detection success rate are consistent metrics across the 8-day history and are shown above. Historical avg_tokens values mix raw tokens (earlier days) with the AIC proxy (recent days), so the token axis is intentionally omitted from the trend to avoid a misleading unit mismatch. Detection success rate has stayed in the 80–95% band; misconfigured count has trended down (7 → 5 → 5 → 2).

Recommendations

1. Copilot Agent Prompt Clustering Analysis — enable gh-aw-detection: true; as an analysis/clustering workflow it should be in scope for detection.
2. Smoke CI — confirm whether this CI workflow should be excluded from the detection heuristic. If it is genuinely non-agentic (it consumes 0 AIC and is currently failing 0/7 on unrelated CI issues), add it to a detection allowlist/suppression so the "explicitly-disabled active workflow" rule stops firing. Separately, its 0/7 success rate is an unrelated CI-health issue worth investigating.
3. Token telemetry gap — TokenUsage is 0 for every run in this window. If token-based reporting matters, verify the token-accounting step is emitting into run_summary.json/aw_info.json; until then reports fall back to the AIC proxy.
4. Detection health is good overall — enabling detection shows no measurable reliability penalty (81% vs 82% success), so wider adoption on analysis/report/monitor workflows is low-risk.

References:

• §28703236869 — Copilot Agent Prompt Clustering Analysis
• §28704356594 — Smoke CI
• §28707011875 — this report run

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔎 Detection Analysis Report · 188.2 AIC · ⌖ 35.9 AIC · ⊞ 7.2K · ◷

• expires on Jul 7, 2026, 5:15 AM UTC-08:00