[detection-analysis] Detection Analysis Report — 2026-10-23 (last 24h)

[github-actions[bot]]

Summary

• Window evaluated: last 24 full hours (UTC), 2026-10-22T23:21Z → 2026-10-23T23:21Z
• Total runs analyzed: 475 (of 614 collected run directories; 139 were fork/cross-repo PR runs with GitHub metadata only, no aw_info.json/run_summary.json, and were excluded from classification)
• Detection-enabled runs: 423 (89.1%)
• Regular runs (gh-aw-detection: false): 52 (10.9%)
• Misconfigured workflows found: 0

Note

No misconfigured workflows were detected in this window. All 52 gh-aw-detection: false runs belong to one-off smoke/conformance workflows (single run each in this window), none of which crossed the >3-run recurrence threshold that would flag an opt-out as a misconfiguration. No workflow had a failing detection job, and no workflow alternated its detection setting within the window.

Comparison Chart

Metric	Regular Runs	Detection Runs
Run count	52	418 (completed)
Success rate	76.9%	90.0%
Failure count	12	42
Avg token usage	99,731	139,047

(Success rate/avg tokens computed over completed runs only; 5 runs — including this report's own run — were still in_progress at collection time and excluded from rate calculations.)

Misconfigured Workflows

No misconfigured workflows detected in this window.

View All Run Metrics

229 distinct workflows ran in the window. Top 20 by run count:

Workflow	Runs	Detection	Success Rate	Avg Tokens
PR Sous Chef	82	enabled	100.0%	164
Issue Monster	47	enabled	97.8%	97,743
Avenger	17	enabled	94.1%	255,188
Ponytail Reviewer	15	enabled	100.0%	97,220
Design Decision Gate 🏗️	14	enabled	100.0%	1,222
Matt Pocock Skills Reviewer	14	enabled	100.0%	34,748
Test Quality Sentinel	14	enabled	100.0%	10,020
PR Code Quality Reviewer	13	enabled	100.0%	16,200
Daily Go Test Parallelizer	12	enabled	91.7%	996,604
AI Moderator	6	enabled	100.0%	952
Contribution Check	6	enabled	100.0%	16,168
Auto-Triage Issues	5	enabled	100.0%	136,657
PR Triage Agent	4	enabled	100.0%	51,885
Code Scanning Fixer	4	enabled	100.0%	10,994
[aw] Failure Investigator (6h)	4	enabled	75.0%	312
Release	3	enabled	100.0%	7,689
Daily Credit Limit Test	2	enabled	100.0%	23,088
ESLint Monster	2	enabled	0.0%	124,360
Cache directory setup	1	enabled	100.0%	80,474
Daily Documentation Healer	1	enabled	0.0%	2,080

Remaining 209 workflows (1 run each) account for 209 runs, almost entirely one-off "Smoke ..." engine-conformance checks — this is where all 52 gh-aw-detection: false runs live (e.g. Smoke Copilot, Smoke Codex, Smoke Claude, Engine Conformance Claude/Codex/Agy). These are expected to opt out of detection by design (short-lived CI smoke tests), not misconfigurations.

Top failure signals (all runs, not detection-specific): the dominant failure driver in this window was the activation job step "Restore daily AIC scan observations" (driver_exit, 8 runs) and downstream conclusion-job aftermath signals (22 runs) that cascade from earlier failures rather than originating in the detection job. No detection-job-specific failures were observed anywhere in the 475 analyzed runs.

Coverage note: the initial collection hit a 3,000-item count limit and returned a continuation pointing at a narrow ~3-minute gap (2026-10-22T23:20:58Z–23:24:16Z, before run §39871540834). A follow-up fetch for that exact gap returned 0 additional runs, so coverage for the full 24h window is effectively complete.

View Historical Trend

25 historical data points now stored in cache-memory (trending/detection-metrics/history.jsonl), covering 13 distinct days over the last 30. Detection-run volume has grown roughly 4x since 2026-10-07 while detection success rate has stayed in the 75–95% band, tracking overall run volume growth rather than detection-specific issues.

Recommendations

1. No action required for detection misconfiguration — the feature is enabled by default across the fleet and no workflow shows a broken detection job or inconsistent opt-out/opt-in behavior this window.
2. Leave the one-off smoke/conformance workflows as-is — their gh-aw-detection: false setting is appropriate for short, single-run CI checks and doesn't meet the recurrence bar for flagging.
3. Investigate the activation job failure pattern separately — "Restore daily AIC scan observations" failed in 8 runs across multiple workflows this window (e.g. run §39936302841). This is an infra/cache-restore issue upstream of both regular and detection paths, not a detection-feature defect, but it's the single largest failure cluster in the window and worth its own investigation.
4. ESLint Monster (0% success, 2/2 runs) and Daily Documentation Healer (0% success, 1/1 run) had no successful runs this window; worth a quick look, though out of scope for this detection-focused report.

References:

• §39871540834 — boundary of the resolved coverage gap
• §39936302841 — example of the dominant activation-job failure cluster
• §39934288281 — representative failed run from the audit's failure-investigation sample

Generated by 🔎 Detection Analysis Report · claude · agent · ~151 AIC · ⌖ ~1 AIC · ⊞ 6.7K · ◷

• expires on Oct 26, 2026, 3:41 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #71716.