[detection-analysis] Detection Analysis Report — 2026-10-12

[github-actions[bot]]

Summary

• Intended window: last 24 full hours ending at workflow start (UTC), i.e. approximately 2026-10-11T23:31:16Z → 2026-10-12T23:31:16Z
• Coverage caveat: pre-downloaded logs actually span only 2026-10-12T14:53:06Z → 2026-10-12T23:22:06Z (~8.5 hours), not the full 24-hour window. Metrics below reflect this narrower subset only.
• Run directories collected: 312; successfully parsed with full metadata (aw_info.json + run_summary.json): 231 (74%); incomplete (metadata missing, likely failed before artifact upload): 81 (26%) — these could not be classified by detection status and are excluded from the metrics below.
• Total runs analyzed: 231
• Detection-enabled runs: 203 (87.9%)
• Regular runs (detection explicitly disabled): 28 (12.1%)
• Misconfigured workflows found: 3

Warning

3 workflows were flagged for explicitly disabling gh-aw-detection while running at volume. See below — this may be an intentional choice for fast CI smoke workflows, but should be confirmed.

Comparison Chart

Metric	Regular Runs	Detection Runs
Run count	28	203
Success rate	92.9%	82.8%
Failure count	2	35
Avg. token usage	8,909	97,389

Detection-enabled runs use substantially more tokens on average (~11x), consistent with the added detection analysis step, and show a lower success rate — driven mainly by a handful of high-token workflows (e.g. "PR Code Quality Reviewer", "Weekly Issue Summary") rather than detection itself causing failures (no detection job in this window had a failing conclusion — see Rule 2 below).

Misconfigured Workflows

Workflow	Misconfiguration Type	Run Count	Recommended Fix
Smoke Claude Routed	gh-aw-detection: false on high-volume workflow (>3 runs in window)	10	Confirm this is an intentional opt-out for a fast CI smoke test; if not, remove gh-aw-detection: false to restore default detection. Example: §38276259477
Smoke Copilot SDK Routed	gh-aw-detection: false on high-volume workflow (>3 runs in window)	9	Same as above. Example: §38301616675
Smoke Pi Routed	gh-aw-detection: false on high-volume workflow (>3 runs in window)	9	Same as above. Example: §38301616685

No workflow showed a failing detection job conclusion (Rule 2), and no workflow alternated between detection-enabled and detection-disabled within the window (Rule 3).

View All Run Metrics

Workflow	Detection Runs	Regular Runs	Success Rate	Avg Tokens
PR Sous Chef	31	0	93.5%	2,007
Auto-Triage Issues	15	0	100.0%	108,156
Design Decision Gate 🏗️	13	0	100.0%	1,585
Matt Pocock Skills Reviewer	13	0	100.0%	36,185
PR Code Quality Reviewer	13	0	61.5%	761,346
Ponytail Reviewer	13	0	100.0%	123,693
Test Quality Sentinel	13	0	100.0%	8,456
Issue Monster	12	0	91.7%	73,088
Smoke Claude Routed	0	10	100.0%	1,858
Smoke Copilot Routed	10	0	100.0%	95,620
Daily Trajectory Grader Implementer	9	0	100.0%	7,209
Smoke Copilot SDK Routed	0	9	100.0%	2,450
Smoke Pi Routed	0	9	77.8%	23,202
AI Moderator	8	0	87.5%	790
Daily Go Test Parallelizer	4	0	0.0%	25,503
(46 more workflows with ≤2 runs each — omitted for brevity)

Full per-run data: /tmp/gh-aw/python/data/all_runs.json (workflow run artifact, not persisted beyond this run).

View Historical Trend

Trend built from 15 historical snapshots in cache-memory (today's snapshot appended); only 4 fall within the last 30 days due to a gap in prior collection between mid-September and October 7.

Recommendations

1. Confirm intent for the 3 "Smoke * Routed" workflows — they run frequently (9-10 runs in this ~8.5h window alone) with gh-aw-detection: false. If this is deliberate (e.g. to keep CI smoke tests fast/cheap), document it in the workflow frontmatter with a comment; otherwise re-enable detection.
2. Investigate high-token outliers among detection-enabled runs — "PR Code Quality Reviewer" (761K avg tokens, 61.5% success) and "Weekly Issue Summary" (877K tokens) are driving up the detection-run average and may benefit from prompt/context trimming regardless of detection status.
3. Re-run this analysis over a true 24h window once log collection coverage is confirmed — the current pass only captured ~8.5 hours and 74% of run directories had full metadata; the remaining 26% (81 runs, mostly pull_request and schedule triggered) lacked aw_info.json/run_summary.json, likely because the agent job failed before producing those artifacts.

Generated by 🔎 Detection Analysis Report · claude · agent · 146 AIC · ⊞ 6.5K · ◷

• expires on Oct 15, 2026, 3:33 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #68551.