[detection-analysis] Detection Analysis Report — 2026-11-06

[github-actions[bot]]

Summary

• Window evaluated: last 24 full hours ending 2026-11-06T23:11Z (UTC), back to 2026-11-05T23:14Z
• Total runs discovered in window: 945 (634 with full classification data + 311 metadata-only runs lacking engine/feature detail, see note below)
• Detection-enabled runs: 629 (99.2% of classified runs)
• Regular runs (gh-aw-detection: false): 5 (0.8% of classified runs)
• Misconfigured workflows found: 1

Warning

1 workflow (daily-github-docs-seo-optimizer) has gh-aw-detection: false on a daily-scheduled workflow, which exceeds the >3-runs-per-7-days threshold for rule 1.

Comparison Chart

Regular runs (5) skew heavily toward small one-off test workflows, so the comparison is not apples-to-apples at this sample size — see caveats below.

Misconfigured Workflows

Workflow	Misconfiguration Type	Run Count (24h)	Recommended Fix
Daily GitHub Docs SEO Optimizer (.github/workflows/daily-github-docs-seo-optimizer.md)	Rule 1: gh-aw-detection: false on a workflow scheduled daily (projects to ~7 runs/7 days, exceeding the >3 threshold)	1 (in window)	Remove the gh-aw-detection: false override at line 37 (or confirm intentionally and document why detection should stay off for this workflow), then run make recompile

Example run: §42164372699

Not flagged (reviewed and excluded):

• engine-conformance-copilot / -claude / -aider — gh-aw-detection: false is explicit in frontmatter, but these are workflow_dispatch-only (no schedule), so they don't meet the >3-runs/7-days bar. Likely intentional for the conformance test suite.
• smoke-work-queue — scheduled every 2 days (~3–4 runs/7 days); borderline and not clearly over the threshold, so not flagged.
• No runs were found where an enabled detection job itself failed (rule 2), and no workflow alternated between detection-enabled and detection-disabled within the window (rule 3).

View All Run Metrics

Aggregate (classified runs only, n=634):

Group	Runs	Success Rate	Failures	Avg Tokens
Regular (gh-aw-detection: false)	5	100.0%	0	38,983
Detection-enabled (default/true)	629	90.4%	60	322,828

Per-workflow breakdown (workflows with >1 run in window; 29 of 174 total workflows seen — the remaining 145 each had exactly 1 run):

Workflow	Runs	Detection Status	Success Rate	Avg Tokens
PR Sous Chef	126	Detection	97.6%	1,177
Test Quality Sentinel	46	Detection	100.0%	137,637
Matt Pocock Skills Reviewer	46	Detection	91.3%	81,240
PR Specification Reviewer	45	Detection	95.6%	3,105
Ponytail Reviewer	44	Detection	100.0%	554,601
PR Code Quality Reviewer	44	Detection	100.0%	16,685
Design Decision Gate 🏗️	42	Detection	83.3%	40,712
Daily Go Test Parallelizer	12	Detection	91.7%	6,947,640
Avenger	11	Detection	81.8%	1,029,150
Auto-Triage Issues	11	Detection	100.0%	213,732
AI Moderator	10	Detection	100.0%	911
Contribution Check	6	Detection	100.0%	30,338
[aw] Failure Investigator (6h)	5	Detection	80.0%	379
PR Triage Agent	4	Detection	100.0%	57,428
Code Scanning Fixer	4	Detection	50.0%	18,328
Visual Regression Checker	4	Detection	75.0%	8,226
Issue Monster	3	Detection	100.0%	107,440
Smoke Work Queue Dispatcher	3	Detection	66.7%	7,004
Smoke Work Queue Worker	3	Detection	33.3%	5,015
Daily Credit Limit Test	2	Detection	50.0%	58,188
Smoke Claude Dynamic Workflow	2	Detection	50.0%	1,341
Daily Documentation Diagram	2	Detection	100.0%	888,329
Sighthound Security Scan	2	Detection	100.0%	197,855
Schema Consistency Checker	2	Detection	100.0%	887,285
Daily Grader Audit	2	Detection	100.0%	27,828
Daily AW Cross-Repo Compile Check	2	Detection	100.0%	34,116
Daily Spending Forecast	2	Detection	50.0%	1,310,995
Copilot Agent Prompt Clustering Analysis	2	Detection	100.0%	54,741
Issue Triage Agent	2	Detection	100.0%	1,117,520

Note on coverage: 311 of 945 runs in the window were "metadata-only" (only the GitHub Actions run record was available, no aw_info.json/run_summary.json), so their detection status and token usage could not be determined. These are excluded from the metrics above and from misconfiguration rules, per the grading contract's handling of metadata_only groups.

View Historical Trend

Trend history now has 38 daily data points (/tmp/gh-aw/cache-memory/trending/detection-metrics/history.jsonl), covering a 30-day window with data on 25 of those days.

Recommendations

1. Fix daily-github-docs-seo-optimizer: remove the gh-aw-detection: false override (or add a comment explaining why detection must stay disabled for a daily-scheduled workflow) and run make recompile.
2. The detection-enabled success rate (90.4%) trails the (very small, n=5) regular-run sample (100%) mainly because the detection group includes nearly all production traffic, including high-churn workflows like PR Sous Chef (126 runs) and several high-token research workflows with intermittent failures (e.g. Avenger, Design Decision Gate, Daily Go Test Parallelizer). None of these failures were attributed to the detection job itself — all failures occurred in the agent job, consistent with normal agent-logic failures rather than a detection feature defect.
3. Re-evaluate smoke-work-queue (every-2-days schedule, gh-aw-detection not set) in a future window with more data; it sits close to the >3-runs/7-days boundary and is worth revisiting as more runs accumulate.

References:

• §42164372699 — Daily GitHub Docs SEO Optimizer (misconfigured workflow example)
• §42193563559 — Daily MCP Tool Concurrency Analysis (sample detection-enabled run)

Generated by 🔎 Detection Analysis Report · claude · agent · ~114 AIC · ⌖ ~1 AIC · ⊞ 4.5K · ◷

• expires on Nov 9, 2026, 3:32 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #75767.