[detection-analysis] Detection Analysis Report — 2026-07-08 (last 24h)

[github-actions[bot]]

Summary

• Window evaluated: last 24 full hours (UTC) — captured runs span 2026-07-08T07:13:40Z → 2026-07-08T12:59:42Z
• Total runs analyzed: 97
• Detection-enabled runs: 37 (38%)
• Regular runs: 60 (62%)
• Misconfigured workflows found: 7

Warning

7 workflows show detection-related misconfigurations. Detection-enabled runs have a higher success rate (83.8%) than regular runs (73.3%), but several audit/analysis workflows are missing the gh-aw-detection: true flag, and two active workflows (Smoke CI, Avenger) are failing consistently.

Note

The logs download timed out mid-transfer, so this is a partial capture (97 of the runs in the window carried complete metadata). Figures reflect the captured subset. Misconfiguration criterion #1 (>3 runs over 7 days) is approximated against the captured window since only a partial 7-day history was available.

Comparison Chart

Metric	Regular Runs	Detection Runs
Total runs	60	37
Success rate	73.3%	83.8%
Avg tokens (in+out+cache)	1,074,444	1,131,936
Failure count	16	6
Misconfigured workflows	—	7

Misconfigured Workflows

Workflow	Misconfiguration type	Runs	Recommended fix
Smoke CI	Detection explicitly disabled on active workflow (gh-aw-detection: false, 8 runs, all failing)	8	Investigate the failing job; re-enable gh-aw-detection: true unless there is a documented reason to keep it off.
Avenger	Detection-enabled run failures (5 of 6 runs failed; ErrorCount 0 at agent level ⇒ likely job/detection-step failure)	5	Inspect the detection job step and job setup for run 28924672312; rerun after fixing the underlying failure.
Daily Max Ai Credits Test	Detection-enabled run failure (1/1 failed)	1	Review run 28940109928 for detection-job errors.
Copilot Agent Prompt Clustering Analysis	Detection feature absent on audit/analysis workflow	1	Add gh-aw-detection: true so detection safeguards apply.
Typist - Go Type Analysis	Detection feature absent on analysis workflow	1	Add gh-aw-detection: true to this analysis workflow.
Daily Agentic Workflow AIC Usage Audit	Detection feature absent on audit workflow	1	Add gh-aw-detection: true to this audit workflow.
GitHub MCP Structural Analysis	Detection feature absent on analysis workflow	1	Add gh-aw-detection: true to this analysis workflow.

View All Run Metrics

Detection column: on = all runs gh-aw-detection: true; off = explicitly false; absent = flag unset; mixed = inconsistent across runs.

Workflow	Runs	Success	Success %	Detection	Avg tokens
AI Moderator	6	6	100%	on	190,678
Auto-Triage Issues	3	3	100%	on	30,471
Avenger	6	1	17%	on	993,818
Claude Code User Documentation Review	1	1	100%	on	3,465,421
Code Scanning Fixer	1	1	100%	on	66,204
Constraint Solving — Problem of the Day	1	1	100%	on	253,387
Contribution Check	1	1	100%	on	1,754,491
Copilot Agent Prompt Clustering Analysis	1	1	100%	absent	1,271,639
Copilot PR Conversation NLP Analysis	1	1	100%	on	1,284,069
Copilot Session Insights	1	1	100%	on	2,674,163
Daily AIC Consumption Report (Sentry + Grafana OTel)	1	1	100%	on	1,723,026
Daily AW Cross-Repo Compile Check	1	1	100%	on	2,718,429
Daily Agentic Workflow AIC Usage Audit	1	1	100%	absent	660,648
Daily Choice Type Test	1	1	100%	on	152,373
Daily Documentation Updater	1	1	100%	on	453,525
Daily Go Function Namer	1	1	100%	on	106,547
Daily MCP Tool Concurrency Analysis	1	1	100%	on	1,615,538
Daily Max Ai Credits Test	1	0	0%	on	19,427
Daily News	1	1	100%	on	974,820
Daily Rendering Scripts Verifier	1	1	100%	on	4,394,635
Daily Syntax Error Quality Check	1	1	100%	absent	148,948
Daily Workflow Updater	1	1	100%	absent	618,129
Dependabot Dependency Checker	1	1	100%	absent	1,886,875
Design Decision Gate 🏗️	4	4	100%	absent	137,254
Dev	1	1	100%	absent	400,675
Draft PR Cleanup	1	1	100%	absent	79,082
ESLint Miner	1	1	100%	absent	3,957,522
GitHub API Consumption Report Agent	1	1	100%	on	2,949,275
GitHub MCP Structural Analysis	1	1	100%	absent	1,588,679
Glossary Maintainer	1	1	100%	absent	3,198,340
Go Fan	1	1	100%	absent	684,643
Impeccable Skills Reviewer	4	3	75%	absent	843,484
Instructions Janitor	1	1	100%	absent	3,940,895
Issue Monster	4	4	100%	absent	42,532
Matt Pocock Skills Reviewer	4	4	100%	absent	999,618
PR Code Quality Reviewer	4	2	50%	absent	1,543,438
PR Description Updater	4	4	100%	absent	590,400
PR Sous Chef	8	4	50%	absent	319,252
PR Triage Agent	1	1	100%	absent	782,236
Package Specification Enforcer	1	1	100%	absent	143,415
Package Specification Extractor	1	1	100%	absent	6,531,345
Release	1	1	100%	absent	247,498
Smoke CI	8	0	0%	off	0
Smoke Copilot	1	0	0%	off	2,799,199
Team Status	1	1	100%	absent	463,034
Terminal Stylist	1	1	100%	absent	1,518,866
Test Quality Sentinel	4	4	100%	on	870,669
Typist - Go Type Analysis	1	1	100%	absent	2,747,742
[aw] Failure Investigator (6h)	1	1	100%	on	1,137,306
daily-experiment-report	1	1	100%	on	4,767,340

View Historical Trend

12 days of history (2026-06-27 → 2026-07-08). Detection-enabled run counts have trended upward (0 → 37/day) while the detection success-rate band has held in the 80–95% range. Regular-run success rate dipped to 73.3% today — the lowest in the window — driven largely by the fully-failing Smoke CI runs.

Recommendations

1. Fix the failing active workflows first. Smoke CI (8/8 failed, detection off) and Avenger (5/6 failed, detection on) are the largest contributors to today's failure count. Their agent-level ErrorCount is 0, pointing at job-level or detection-step failures rather than agent logic — inspect the workflow job configuration.
2. Enable detection on audit/analysis workflows. Four report/analysis-style workflows (Copilot Agent Prompt Clustering Analysis, Typist - Go Type Analysis, Daily Agentic Workflow AIC Usage Audit, GitHub MCP Structural Analysis) run without gh-aw-detection: true. Add the flag in frontmatter and recompile the .lock.yml.
3. Detection is not hurting reliability. Detection-enabled runs outperform regular runs on success rate (83.8% vs 73.3%) at a modest +5% average-token cost — supporting broader adoption.
4. Re-run with a longer logs timeout to capture the full 24h set; today's partial capture may under-count failures and misconfigurations.

References:

• §28945283417 (this report)
• §28929526023 (Smoke CI failure)
• §28924672312 (Avenger detection failure)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔎 Detection Analysis Report · 203.8 AIC · ⌖ 36.5 AIC · ⊞ 7.1K · ◷

• expires on Jul 11, 2026, 5:26 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #44557.