[detection-analysis] Detection Analysis Report — 2026-07-17 (last 24h)

[github-actions[bot]]

Summary

• Window evaluated: last 24 full hours (UTC) — 2026-07-16T13:20:38Z → 2026-07-17T13:20:38Z
• Total runs analyzed: 157 (runs with complete aw_info.json + run_summary.json)
• Detection-enabled runs (gh-aw-detection: true): 45 (28.7%)
• Regular runs (detection false/absent): 112 (71.3%)
• Misconfigured workflows found: 8

Warning

8 misconfigured workflows were detected in this window. See Misconfigured Workflows below for the specific findings and recommended fixes.

Detection-enabled runs had a higher success rate (91.1% vs 79.5%) but consumed roughly 2x the average tokens (147,569 vs 70,776) — consistent with detection performing additional analysis steps.

Comparison Chart

Stacked bars show success/failure counts per group; the blue line overlays success rate (right axis). The red band flags that misconfigured workflows are present.

Misconfigured Workflows

Workflow	Misconfiguration type	Runs	Example	Recommended fix
Duplicate Code Detector	Name implies detection role (contains detector) but gh-aw-detection not enabled	1	§29558767696	Add gh-aw-detection: true so detection steps run for this analysis workflow.
Daily Agentic Workflow AIC Usage Audit	Name implies detection role (contains audit) but gh-aw-detection not enabled	1	§29578448593	Add gh-aw-detection: true so detection steps run for this audit workflow.
Smoke CI	gh-aw-detection: false on high-frequency workflow (18 runs, >3 threshold)	18	§29559085259	Enable detection, or confirm opt-out is intentional for this CI smoke workflow.
Smoke Copilot MAI	gh-aw-detection: false on high-frequency workflow (4 runs, >3 threshold)	4	§29562782974	Enable detection, or confirm opt-out is intentional.
Avenger	Detection-enabled run failed — detection steps may not have completed	1	§29565417785	Inspect the failed run's detection steps/logs and fix the underlying error.
daily-experiment-report	Detection-enabled run failed — detection steps may not have completed	1	§29566718994	Inspect the failed run's detection steps/logs and fix the underlying error.
Copilot PR Conversation NLP Analysis	Detection-enabled run failed — detection steps may not have completed	1	§29574387247	Inspect the failed run's detection steps/logs and fix the underlying error.
Daily Max Ai Credits Test	Detection-enabled run failed — detection steps may not have completed	1	§29577507257	Inspect the failed run's detection steps/logs and fix the underlying error.

View All Run Metrics

Per-workflow breakdown (81 distinct workflows in the window). Detection column: on = all runs enabled, off = explicitly false, absent = feature unset, mixed = flip-flopped within window.

Workflow	Runs	Detection	Success rate	Fail	Avg tokens
Smoke CI	18	off	0.0%	18	0
PR Sous Chef	9	absent	88.9%	1	31,973
Impeccable Skills Reviewer	7	absent	100.0%	0	6,343
Matt Pocock Skills Reviewer	7	absent	100.0%	0	10,071
PR Code Quality Reviewer	6	absent	83.3%	1	12,361
Test Quality Sentinel	6	on	100.0%	0	8,500
Design Decision Gate 🏗️	6	absent	100.0%	0	8,888
PR Description Updater	6	absent	100.0%	0	13,810
Tidy	5	absent	80.0%	1	5,453
Smoke Copilot MAI	4	off	100.0%	0	104,650
Issue Monster	3	absent	100.0%	0	21,228
AI Moderator	3	on	100.0%	0	42,385
PR Triage Agent	2	absent	50.0%	1	35,574
Avenger	2	on	50.0%	1	0
Auto-Triage Issues	2	on	100.0%	0	25,614
Smoke Copilot	2	off	100.0%	0	1,026,283
Smoke Copilot - AOAI (apikey)	2	off	100.0%	0	203,786
Smoke Copilot - AOAI (Entra)	2	off	100.0%	0	368,586
Smoke Copilot Small	2	off	100.0%	0	3,140
Smoke OTEL	2	off	100.0%	0	16,140
Daily Cli Tools Tester	1	on	100.0%	0	15,635
ESLint Refiner	1	absent	100.0%	0	66,126
Duplicate Code Detector	1	absent	100.0%	0	149,886
Schema Consistency Checker	1	absent	100.0%	0	54,739
CLI Version Checker	1	on	100.0%	0	26,580
GitHub Remote MCP Authentication Test	1	on	100.0%	0	18,758
Daily Sub-Agent Model Resolution Audit	1	on	100.0%	0	3,032,602
Multi-Device Docs Tester	1	on	100.0%	0	71,060
Daily AgentRx Trace Optimizer	1	on	100.0%	0	39,822
Daily Safe Outputs Conformance Checker	1	absent	100.0%	0	15,796
Daily Assign Issue To User	1	on	100.0%	0	4,027
Daily Hippo Learn	1	on	100.0%	0	69,740
Dependabot Burner	1	absent	100.0%	0	119,748
Sub-Issue Closer	1	absent	100.0%	0	7,742
Release	1	absent	100.0%	0	7,999
Copilot Session Insights	1	on	100.0%	0	51,870
Discussion Task Miner - Code Quality Improvement Agent	1	absent	0.0%	1	67,116
[aw] Failure Investigator (6h)	1	on	100.0%	0	21,678
Code Scanning Fixer	1	on	100.0%	0	7,823
Daily Workflow Updater	1	absent	100.0%	0	7,477
Daily Elixir Credo Snippet Audit	1	on	100.0%	0	10,006
Daily Evals Feature Report	1	on	100.0%	0	1,753,325
Daily Syntax Error Quality Check	1	absent	100.0%	0	7,116
Daily Rendering Scripts Verifier	1	on	100.0%	0	29,806
daily-experiment-report	1	on	0.0%	1	68,604
Go Fan	1	absent	100.0%	0	26,347
Instructions Janitor	1	absent	100.0%	0	30,307
Daily News	1	on	100.0%	0	71,445
Terminal Stylist	1	absent	100.0%	0	17,408
ESLint Miner	1	absent	100.0%	0	31,724
Daily AW Cross-Repo Compile Check	1	on	100.0%	0	43,153
Dev	1	absent	100.0%	0	487,159
Contribution Check	1	on	100.0%	0	27,071
Daily MCP Tool Concurrency Analysis	1	on	100.0%	0	10,120
Dependabot Dependency Checker	1	absent	100.0%	0	8,790
Package Specification Extractor	1	absent	100.0%	0	13,477
Copilot Agent Prompt Clustering Analysis	1	on	100.0%	0	45,474
Glossary Maintainer	1	absent	100.0%	0	18,270
Copilot PR Conversation NLP Analysis	1	on	0.0%	1	104,058
Daily Documentation Updater	1	on	100.0%	0	64,021
GitHub API Consumption Report Agent	1	on	100.0%	0	48,426
Draft PR Cleanup	1	absent	100.0%	0	5,425
Smoke Codex	1	off	100.0%	0	280,360
Changeset Generator	1	on	100.0%	0	0
Smoke Gemini	1	absent	100.0%	0	374,304
Smoke Claude	1	off	100.0%	0	22,526
Agent Container Smoke Test	1	off	100.0%	0	3,956
Smoke Pi	1	off	100.0%	0	12,419
Smoke Antigravity	1	off	100.0%	0	0
Daily Choice Type Test	1	on	100.0%	0	6,635
Daily Max Ai Credits Test	1	on	0.0%	1	2,851
Typist - Go Type Analysis	1	on	100.0%	0	117,628
Visual Regression Checker	1	absent	100.0%	0	15,951
Package Specification Enforcer	1	absent	100.0%	0	83,090
Constraint Solving — Problem of the Day	1	on	100.0%	0	7,868
Team Status	1	absent	100.0%	0	10,069
Daily Agentic Workflow AIC Usage Audit	1	absent	100.0%	0	11,329
Daily Go Function Namer	1	on	100.0%	0	31,040
Daily AIC Consumption Report (Sentry + Grafana OTel)	1	on	100.0%	0	46,560
GitHub MCP Structural Analysis	1	on	100.0%	0	32,527
Claude Code User Documentation Review	1	on	100.0%	0	88,313

View Historical Trend

20 days of history recorded in cache-memory trending. Detection run volume has trended upward while detection success rate holds in the ~85–95% band.

Recommendations

1. Enable detection on analysis-oriented workflows. Duplicate Code Detector and Daily Agentic Workflow AIC Usage Audit have detector/audit roles but run without gh-aw-detection: true. Add the flag so their outputs get detection coverage.
2. Confirm intent for high-frequency detection-off workflows. Smoke CI (18 runs, all short-circuit / 0 tokens) and Smoke Copilot MAI (4 runs) run detection-disabled at volume. These are likely smoke/CI harnesses where opt-out is intentional — annotate the config to make the opt-out explicit and suppress future flags.
3. Investigate the 4 failed detection-enabled runs. Avenger, daily-experiment-report, Copilot PR Conversation NLP Analysis, and Daily Max Ai Credits Test failed with detection enabled; verify whether the detection steps or the workflow body caused the failure.
4. Watch token cost. Detection runs average ~2x tokens; keep an eye on the highest-cost detection workflows (e.g. Daily Sub-Agent Model Resolution Audit, Daily Evals Feature Report) to ensure detection value justifies spend.

Data completeness note: 393 run directories were fetched for the window but only 157 had complete artifacts (aw_info.json + run_summary.json); the remainder were empty (artifact download incomplete). Metrics reflect the 157 complete runs. The >3 runs in 7 days misconfiguration threshold was approximated using the available window since only 24h of logs were downloaded.

References:

• §29558767696
• §29559085259
• §29565417785

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔎 Detection Analysis Report · 221.6 AIC · ⌖ 36.7 AIC · ⊞ 6.5K · ◷

• expires on Jul 20, 2026, 5:28 AM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #46440.