[detection-analysis] Detection Analysis Report — 2026-08-15 (24h)

[github-actions[bot]]

Summary

• Window evaluated: last 24 full hours (2026-08-14T23:21:09Z → 2026-08-15T23:21:09Z, UTC)
• Total runs analyzed: 29 classified (30 log directories captured; 1 run ID, 31913054444, returned "not found" via the API and was excluded — data gap, not a misconfiguration)
• Detection-enabled runs: 19 (65.5%) — 17 completed, 2 still in_progress at analysis time (this report's own run, and "Agent Job Health Monitor")
• Regular runs (detection absent): 10 (34.5%)
• Misconfigured workflows found: 8

Warning

8 misconfigured workflows found relative to the gh-aw-detection feature — see table below.

Important caveat on the "regular" bucket: none of the 10 regular-classified runs in this window are actual detection failures. All 10 belong to three workflows (AI Moderator ×5, Q ×2, PR Description Updater ×3) whose source (.md) frontmatter already sets gh-aw-detection: true — they simply never reached the agent-execution step in this run instance (conclusion: action_required, 0 tokens, 0 turns), most likely gated on required-reviewer approval for bot/fork-originated PRs. They are bucketed as "regular" here strictly because aw_info.json (and therefore features.gh-aw-detection) was never written for these runs, per the classification rule of using features.gh-aw-detection directly rather than the lock file. Treat the 0% "success rate" for Regular Runs below as "gated, pending approval" — not a detection defect.

Comparison Chart

Misconfigured Workflows

Workflow	Misconfiguration Type	Run Count	Recommended Fix
Daily GitHub Docs SEO Optimizer (daily-github-docs-seo-optimizer.md)	Rule 1: gh-aw-detection: false on a daily-scheduled workflow (~7 runs/7d expected; 0 in this 24h window) with unrestricted bash: ["*"] and issue-creation safe-output	~7/week (schedule: daily)	Remove the gh-aw-detection: false override (or set to true) given its broad bash access and untrusted-content (GitHub Docs) processing
GitHub MCP Tools Report (github-mcp-tools-report.md)	Rule 2: name contains "report" but lacks gh-aw-detection: true	0 in window (schedule: weekly, Sun 12:00)	Add features: { gh-aw-detection: true }
MCP Inspector (mcp-inspector.md)	Rule 2: name contains "inspector" but lacks gh-aw-detection: true	0 in window (schedule: weekly, Mon 18:00)	Add features: { gh-aw-detection: true }
Objective Impact Report (objective-impact-report.md)	Rule 2: name contains "report" but lacks gh-aw-detection: true	0 in window (workflow_dispatch only, private)	Add features: { gh-aw-detection: true }
Org Health Report (org-health-report.md)	Rule 2: name contains "report" but lacks gh-aw-detection: true	0 in window (schedule: weekly, Mon 09:00)	Add features: { gh-aw-detection: true }
Repo Audit Analyzer (repo-audit-analyzer.md)	Rule 2: name contains "audit"/"analyzer" but lacks gh-aw-detection: true	0 in window (workflow_dispatch only, private, targets external repos e.g. FStarLang/FStar)	Add features: { gh-aw-detection: true } — especially relevant since it audits third-party repo content
Ubuntu Actions Image Analyzer (ubuntu-image-analyzer.md)	Rule 2: name contains "analyzer" but lacks gh-aw-detection: true	0 in window (schedule: weekly)	Add features: { gh-aw-detection: true }
Video Analyzer (video-analyzer.md)	Rule 2: name contains "analyzer" but lacks gh-aw-detection: true	0 in window (workflow_dispatch only, private, accepts arbitrary user-supplied video_url)	Add features: { gh-aw-detection: true } — user-supplied URL input increases relevance of detection

agentic-token-audit.md ("Daily Agentic Workflow AIC Usage Audit") matched the "audit" name pattern but is excluded per the documented repository-level opt-out (source-managed from githubnext/agentic-ops).

smoke-*.md workflows (36 files) also set gh-aw-detection: false but are intentional smoke tests exercising the no-detection code path; none ran in this window and they are not flagged here.

No rule-3 (detection steps failed on a gh-aw-detection: true run) or rule-4 (a single workflow alternating between detection-enabled/disabled within the window) violations were found. An initial read suggested "Design Decision Gate 🏗️" alternated between enabled/disabled, but re-fetching full run data confirmed both of its runs in this window (31914006955, 31914018000) have gh-aw-detection: true — the first run's aw_info.json simply wasn't present in the initial log pre-download and needed an explicit re-fetch.

View All Run Metrics

Detection-enabled runs (completed, n=17):

Workflow	Engine	Tokens	Run
Issue Monster	pi	16,568	§31912846670
PR Sous Chef	pi	49,037	§31912931553
PR Sous Chef	pi	24,058	§31913499533
Test Quality Sentinel	copilot	6,233	§31914006953
Design Decision Gate 🏗️	claude	1,227,370 ⚠️ outlier	§31914006955
Ponytail Reviewer	copilot	5,778	§31914006972
PR Code Quality Reviewer	pi	33,400	§31914006988
Matt Pocock Skills Reviewer	copilot	10,224	§31914007011
Impeccable Skills Reviewer	copilot	5,088	§31914007047
Impeccable Skills Reviewer	copilot	4,860	§31914017988
Ponytail Reviewer	copilot	5,042	§31914017989
Test Quality Sentinel	copilot	971,534 ⚠️ outlier	§31914017991
Matt Pocock Skills Reviewer	copilot	9,151	§31914017997
Design Decision Gate 🏗️	claude	9,335	§31914018000
PR Code Quality Reviewer	pi	18,657	§31914018006
Issue Monster	pi	15,291	§31914092630
PR Sous Chef	pi	~48,837	§31914206398

Still in_progress (excluded from rate/token calcs): Detection Analysis Report (§31914092917 — this report's own run), Agent Job Health Monitor (§31914141926).

Regular / detection-absent runs (all gated, conclusion action_required, 0 tokens):

Workflow	Count	Example Run
AI Moderator	5	§31912821654
Q	2	§31912935842
PR Description Updater	3	§31913214793

Engine mix (detection-enabled runs): copilot 8, pi 6, claude 2 (+1 in-progress copilot, +1 in-progress claude)

Two token-usage outliers were observed among detection-enabled runs — Design Decision Gate 🏗️ (1,227,370 tokens, run §31914006955) and Test Quality Sentinel (971,534 tokens, run §31914017991) — roughly 25-100x their peer runs. Not a gh-aw-detection misconfiguration, but worth a separate look at cost/runaway-loop risk.

View Historical Trend

This is the first day this trending metric has been recorded (/tmp/gh-aw/cache-memory/trending/detection-metrics/history.jsonl — 1 data point). A trend chart will be generated once at least 7 daily data points are available.

Recommendations

1. Enable gh-aw-detection: true on the 7 audit/analyzer/report/inspector-named workflows listed above — several of them (Repo Audit Analyzer, Video Analyzer) process third-party or user-supplied content, which is exactly the scenario detection is meant to cover.
2. Re-evaluate whether daily-github-docs-seo-optimizer.md still needs gh-aw-detection: false — it runs daily with unrestricted bash: ["*"] access and no github: tool restriction, a combination that benefits from detection.
3. Investigate why AI Moderator, Q, and PR Description Updater are landing on action_required across all 10 observed runs in this window — if this is expected (e.g., required-reviewer approval for bot-authored PRs), no action is needed, but if it's blocking legitimate moderation/description-update work it may need a workflow-permissions review.
4. Take a quick look at the two 900k+/1.2M-token outlier runs (Design Decision Gate 🏗️, Test Quality Sentinel) for possible runaway loops, independent of detection status.

References:

• §31914006955 — Design Decision Gate 🏗️, token-usage outlier
• §31914017991 — Test Quality Sentinel, token-usage outlier
• §31912821654 — AI Moderator, representative gated/action_required run

Generated by 🔎 Detection Analysis Report · agent · 189.3 AIC · ⌖ 7.38 AIC · ⊞ 7.7K · ◷

• expires on Aug 18, 2026, 3:36 PM UTC-08:00

[github-actions[bot]]

This discussion has been marked as outdated by Detection Analysis Report.

A newer discussion is available at Discussion #53241.