[detection-analysis] Detection Analysis Report — 2026-09-12

[github-actions[bot]]

Summary

• Window evaluated: last 24 full hours (UTC), 2026-09-11T23:18Z → 2026-09-12T23:18Z
• Total runs analyzed: 426 (of 578 run directories found; 152 lacked agentic-workflow telemetry — likely non-gh-aw CI runs — and were excluded)
• Detection-enabled runs: 425 (99.8%)
• Regular runs (gh-aw-detection: false): 1 (0.2%)
• Misconfigured workflows found (per the detection-specific rules below): 0

Warning

No gh-aw-detection misconfigurations were found, but the data surfaces a bigger, repo-wide problem: 54.7% of all runs in the window (233/426) never reached the agent job because the activation job's "Check daily workflow token guardrail" step failed. This is not a detection-feature bug, but it dominates the failure numbers below and is likely why the "Detection Runs" success rate looks so low. See Recommendations.

Comparison Chart

Regular runs (1 total) aren't visible as a bar at this scale; the line shows their 100% success rate vs. 35.1% for detection-enabled runs. Average token usage was 0 for both groups — TokenUsage is reported as 0 across every single run in this window (success and failure alike), so it isn't included as a meaningful comparison; see Recommendations.

Misconfigured Workflows

No misconfigured workflows detected in this window (no workflow met the "detection disabled at scale," "detection job itself failing," or "inconsistent flag" criteria).

View All Run Metrics

Top workflows by run count (full long tail of 1-run workflows omitted for brevity — 118 additional workflows had exactly 1 run each in the window):

Workflow	Total Runs	Detection	Regular	Success Rate	Avg Tokens
PR Sous Chef	87	87	0	6.9%	0
Issue Monster	46	46	0	0.0%	0
[aw] Failure Investigator (6h)	34	34	0	2.9%	0
Test Quality Sentinel	17	17	0	100.0%	0
Impeccable Skills Reviewer	16	16	0	43.8%	0
Matt Pocock Skills Reviewer	16	16	0	68.8%	0
Design Decision Gate 🏗️	16	16	0	93.8%	0
Ponytail Reviewer	16	16	0	100.0%	0
PR Code Quality Reviewer	16	16	0	0.0%	0
Daily Go Test Parallelizer	12	12	0	58.3%	0
Contribution Check	6	6	0	50.0%	0
Auto-Triage Issues	5	5	0	40.0%	0
Avenger	5	5	0	60.0%	0
PR Triage Agent	4	4	0	0.0%	0
Deep Report	4	4	0	50.0%	0
Code Scanning Fixer	4	4	0	0.0%	0
Release	3	3	0	100.0%	0
AI Moderator	3	3	0	0.0%	0
Daily Credit Limit Test	2	2	0	0.0%	0
Daily GitHub Docs SEO Optimizer	1	0	1	100.0%	0

Root cause behind most of the low success rates above: of the 233 runs whose agent job never ran (skipped, upstream of detection), 230 (98.7%) failed at the same activation step — "Check daily workflow token guardrail" — e.g. §34658100900 (PR Sous Chef) and §34663016855 (Issue Monster). This pattern held steady across all 24 hours of the window (roughly 40–100% of runs per hour hit it), not just a brief spike.

View Historical Trend

Detection-run success rate has been declining across the last five recorded snapshots: 84.6% (Sep 8) → 80.2% (Sep 9) → 76.3% (Sep 10) → 51.2% (Sep 11) → 35.1% (Sep 12), tracking the ramp-up of the activation-job guardrail failures described above. detection_avg_tokens also dropped to 0 starting Sep 11, consistent with the same root cause suppressing normal telemetry.

Recommendations

1. Investigate the "Check daily workflow token guardrail" activation step — it is the single largest driver of failed/skipped runs this window (230 runs, ~54% of all traffic) and is likely responsible for the apparent collapse in the "Detection Runs" success rate and for token usage reporting as 0 everywhere. This looks related to the AIC (agent token/cost) accounting path touched by the most recent commit on main, Handle empty AIC accounting for failed agent runs (#60470) — worth checking whether that change (or a preceding one in the same area) introduced a regression in the guardrail check itself.
2. No action needed on gh-aw-detection configuration specifically — adoption is effectively universal (99.8%) and no workflow shows the disabled/inconsistent/detection-job-failing patterns this report screens for.
3. Once the guardrail issue is resolved, re-run this analysis to confirm the detection success-rate trend recovers back toward the 80–90% baseline seen in late August.

References:

• §34658100900 — PR Sous Chef, activation failed at token guardrail step
• §34663016855 — Issue Monster, same guardrail failure
• §34689997515 — Failure Investigator, agent job failure (non-guardrail)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 🔎 Detection Analysis Report · claude · agent · 219.7 AIC · ⌖ 9.34 AIC · ⊞ 9K · ◷

• expires on Sep 15, 2026, 3:37 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #60704.