[detection-analysis] Detection Analysis Report — 2026-08-12 #52384
Closed
Replies: 1 comment
|
This discussion has been marked as outdated by Detection Analysis Report. A newer discussion is available at Discussion #52595. |
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Summary
Agent Job Health Monitor)Note
No misconfigured workflows detected in this window. See "Misconfigured Workflows" below for a pattern that looked suspicious at first glance but checked out as a telemetry gap, not a real misconfiguration.
Comparison Chart
Regular runs succeeded 4/17 (23.5%) vs. detection runs 10/12 (83.3%). Detection-enabled runs averaged 97,426 tokens; regular runs averaged 0 tokens.
Misconfigured Workflows
No misconfigured workflows detected in this window.
One pattern was investigated and ruled out: AI Moderator (7 runs), Issue Monster (3 runs), and Q (4 runs) each appear in the "Regular" bucket in this window even though their source configs (
ai-moderator.md,issue-monster.md,q.md) all declaregh-aw-detection: true, and their compiled lock files confirmGH_AW_INFO_FEATURES: '{"gh-aw-detection":true}'. This looked like rule-4 alternation at first. On inspection, every one of those "Regular" runs is eitheraction_required(blocked on a GitHub Actions approval gate before the agent job ever started, soaw_info.jsonwas never written) or a 0-turn early-exit "success" (Issue Monster's skip path). None of them reflect an actual toggle of the detection feature — it's a telemetry gap where incomplete/gated runs don't emit detection metadata, not a config problem. No fix needed on the workflow side; flagging this as a possible observability improvement (see Recommendations).AvengerandPR Description Updater/Q/AI Moderator's gated runs make up the rest of the Regular bucket;Avengergenuinely has nogh-aw-detectionfeature configured, but its name doesn't match the audit/analyzer/report/detector/monitor/inspector keyword list, so rule 2 doesn't apply.View All Run Metrics
View Historical Trend
Not enough history yet — this run added the first entry to
/tmp/gh-aw/cache-memory/trending/detection-metrics/history.jsonl. A trend chart will be generated automatically once 7+ daily entries have accumulated.Recommendations
Issue Monster(§31644531765) andDaily BYOK Ollama Test(§31647146503) — each failed with 1 error. Neither failure traced to a detection-specific step in the available logs, but both are worth a closer look.aw_info.json(or equivalent detection-feature marker) earlier in the job — before the approval-gate/early-exit paths — soaction_requiredand 0-turn skip runs don't silently disappear from detection telemetry. This would remove the false "alternation" signal forAI Moderator,Issue Monster, andQgoing forward.gh-aw-detectionconfig changes needed this cycle.References:
All reactions