[detection-analysis] Detection Analysis Report — 2026-11-05 (24h)

[github-actions[bot]]

Summary

• Window evaluated: last 24 full hours (UTC), 2026-11-04 23:28 → 2026-11-05 23:28
• Total runs analyzed: 500 (plus 283 metadata-only runs — superseded/obsolete PR runs with no agent/detection execution data — excluded from classification; see note below)
• Detection-enabled runs: 445 (89.0%)
• Regular runs (gh-aw-detection: false): 55 (11.0%)
• Misconfigured workflows found: 1

Warning

1 workflow showed a detection-job failure in this window (isolated flake, see below). No systemic gh-aw-detection misconfigurations were found.

Comparison Chart

Group	Runs	Success Rate	Failures	Avg Tokens
Regular (detection off)	55	81.8%	10	134,579
Detection-enabled	445	86.5%	60	374,800

Detection-enabled runs show a higher success rate (+4.7 pts) but ~2.8x the average token usage, consistent with the extra detection pass running on top of the agent job for those workflows. The gap is driven partly by a handful of very token-heavy workflows (e.g. Daily Go Test Parallelizer, Ponytail Reviewer) that all run with detection enabled.

Misconfigured Workflows

Workflow	Misconfiguration Type	Run Count	Recommended Fix
PR Code Quality Reviewer	Detection job failing	1	Isolated flake: the detection job's "Install GitHub Copilot CLI" step failed once (§42055218246) while 20 other recent runs of this workflow succeeded with detection passing. Monitor for recurrence rather than treating as a lasting misconfiguration; no frontmatter change needed at this time.

No workflows were flagged for frequently-disabled detection (gh-aw-detection: false on a workflow with >3 runs) — the only runs with detection explicitly disabled were one-off "Smoke" test workflows (20 distinct smoke workflows, 1 run each). No workflows alternated between detection-enabled and detection-disabled within the window.

View All Run Metrics

227 distinct workflows ran in the window. Top 25 by run count:

Workflow	Runs	Detection	Regular	Success %	Avg Tokens	Engines
PR Sous Chef	121	121	0	86.8	3,116	pi
Ponytail Reviewer	22	22	0	100.0	579,414	codex
Matt Pocock Skills Reviewer	21	21	0	100.0	67,318	copilot
PR Code Quality Reviewer	21	21	0	95.2	15,734	copilot
Test Quality Sentinel	21	21	0	100.0	156,605	copilot
Design Decision Gate 🏗️	20	20	0	90.0	121,779	pi
Daily Go Test Parallelizer	12	12	0	100.0	7,069,654	codex
AI Moderator	10	10	0	90.0	876	pi
Avenger	7	7	0	100.0	763,931	copilot
PR Specification Reviewer	7	7	0	71.4	2,578	copilot
Contribution Check	6	6	0	100.0	28,148	copilot
Auto-Triage Issues	5	5	0	100.0	505,993	codex
[aw] Failure Investigator (6h)	5	5	0	80.0	331	claude
PR Triage Agent	4	4	0	100.0	50,713	copilot
Code Scanning Fixer	4	4	0	50.0	18,720	copilot
Smoke Claude Dynamic Workflow	2	2	0	100.0	1,211	claude
Daily Credit Limit Test	2	2	0	100.0	16,146	pi
Cache directory setup	1	1	0	100.0	166,896	opencode
Daily Documentation Healer	1	1	0	0.0	35,275	claude
Daily Model Inventory Checker	1	1	0	100.0	21,294	copilot
Smoke Call Workflow	1	0	1	100.0	36,620	codex
Smoke Copilot Auto	1	0	1	100.0	4,569	copilot
Smoke Workflow Call	1	0	1	100.0	3,424	claude
Smoke Copilot SDK	1	0	1	100.0	1,680	copilot
Smoke Pydantic AI	1	0	1	0.0	216,962	pydantic-ai

(Remaining 202 workflows each had 1 run; mostly additional one-off "Smoke" tests with detection disabled.)

Metadata-only runs excluded: 283 run directories in the window had only minimal run.json metadata (no aw_info.json/run_summary.json — consistent with superseded/obsolete PR-triggered runs cancelled by a newer commit). These could not be classified by detection status and are excluded from the counts above.

View Historical Trend

36 distinct days of history now recorded (with some gaps in the last 30 days). Detection-run volume has grown sharply over the period while regular-run volume stays consistently low (under ~80/day), reflecting that detection is the default and only a small number of one-off smoke tests opt out.

Recommendations

• No frontmatter changes required this cycle — gh-aw-detection usage across the fleet is healthy (89% of runs use the default, remaining 11% are intentional one-off smoke tests).
• Keep an eye on PR Code Quality Reviewer's detection job; if the "Install GitHub Copilot CLI" step fails again on a future run, treat it as a recurring infra issue (e.g. flaky npm registry/network) rather than a one-off.
• Consider investigating why 283 PR-triggered runs in this window were superseded/obsolete before producing execution data — if this ratio is typical, it may be worth tuning concurrency/cancellation settings for high-churn PR workflows to reduce wasted runner minutes (separate from the detection-feature scope of this report).

References:

• §42055218246 — PR Code Quality Reviewer, isolated detection-job failure
• §42004795102 — highest-cluster failure run observed in window (engine-conformance-agy)
• §42004751895 — engine-conformance-codex failure cluster

Generated by 🔎 Detection Analysis Report · claude · agent · ~140 AIC · ⌖ ~1 AIC · ⊞ 4.5K · ◷

• expires on Nov 8, 2026, 3:33 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #75497.