You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Safe Output Messages Failed: 1 (+ 1 cancelled due to cascade)
Safe Output Warnings: 1 (label missing)
Safe Output Success Rate: 95.8%
Detection Failures Blocking Safe Outputs: 4 runs
Error Clusters Identified: 4 clusters
Overall system health is good. The single hard safe output failure (EP019 variant in Smoke Claude) is a known pattern related to protected path enforcement. The 4 detection failures blocking safe outputs are a more significant concern—they silently discard valid agent work.
Safe Output Job Statistics
Job Type
Executions
Failures
Warnings
Success Rate
add_comment
7
0
0
100%
create_discussion
2
0
1
100% (warn)
noop
2
0
0
100%
push_to_pull_request_branch
1
1
0
0%
create_issue
1
0¹
0
100%
dispatch_workflow
1
0
0
100%
set_issue_type
1
0
0
100%
missing_tool
1
0
0
100% (informational)
add_reviewer
1
0
0
100%
update_pull_request
1
0
0
100%
create_pull_request_review_comment
1
0
0
100%
submit_pull_request_review
1
0
0
100%
¹ create_issue was cancelled due to the preceding push_to_pull_request_branch failure.
Runs that executed safe outputs successfully: Issue Monster (×2), Daily Firewall, PR Triage Agent, Constraint Solving, Weekly Editors Health Check, Smoke Copilot, Smoke Claude (partial)
Runs where safe outputs were SKIPPED due to detection failure: Smoke Create Cross-Repo PR, Smoke Update Cross-Repo PR, Auto-Triage Issues, Smoke Gemini
Error: Authentication failed (Request ID: 4004:965CC:C43FAB:FE7892:69C7D214)
##[error]Process completed with exit code 1.
##[error]❌ Failed to parse detection result: No THREAT_DETECTION_RESULT found in detection log.
```
For Smoke Gemini (truncated output):
```
"content": "THREAT_DETECTION_RESULT:{\"prompt_injection\":false,\"secret_leak\":false,\"malicious_",
"delta": true
```
Root Cause: The Copilot detection model either fails to authenticate (3 cases with different request IDs) or produces a streaming response that gets truncated before completing the required output format. The delta:true marker indicates the final message was a streaming delta that was cut off mid-JSON.
Impact: HIGH — 4 workflows had their safe outputs silently blocked. Valid agent outputs (PR patches, issue labels, discussion posts) were discarded.
Pattern History: First observed 2026-03-26 (4 cases), recurring today 2026-03-28 (4 cases). Now 8 total occurrences over 3 days.
##[error]Cannot push to pull request branch: patch modifies protected files (.github/smoke-test-push-23685128220).
Add them to the allowed-files configuration field or set protected-files: fallback-to-issue.
##[error]✗ Message 8 (push_to_pull_request_branch) failed
##[warning]⚠️ Code push operation 'push_to_pull_request_branch' failed — remaining safe outputs will be cancelled
⏭ Message 9 (create_issue) cancelled
Root Cause: Smoke Claude's agent wrote a code patch targeting .github/smoke-test-push-{run_id}, which falls under the protected path prefix .github/. The handler correctly blocks this, but causes cascade cancellation of the following create_issue message.
Impact: MEDIUM — Safe outputs job reports as failed. Issue creation (smoke test result) was not saved.
Cluster 3: EP025 actions/setup Not Found — Infrastructure (Multiple Jobs)
##[error]Can't find 'action.yml', 'action.yaml' or 'Dockerfile' under '/home/runner/work/gh-aw/gh-aw/actions/setup'.
Did you forget to run actions/checkout before running your local action?
Root Cause: This is a checkout ordering issue — the actions/setup local action is referenced before the repository is fully checked out in certain job configurations. Affects push_repo_memory, upload_assets, and agent post-cleanup steps.
Impact: MEDIUM — These are NOT safe output job failures, but they cause workflow conclusion to show failure even when the safe_outputs job itself ran correctly (e.g., PR Triage Agent: safe output noop succeeded, but workflow failed due to push_repo_memory). Also contributes to the detection failures seen in Cluster 1.
Pattern History: First observed 2026-03-26 (2 cases). Now 5 cases today. Escalating.
Affected Workflow: Constraint Solving — Problem of the Day
Sample Warning:
##[warning]Could not find label IDs for: constraint-solving, problem-of-the-day
These labels may not exist in the repository. Available labels: bug, documentation, ...
##[warning]⚠ No matching labels found in repository for: constraint-solving, problem-of-the-day
Root Cause: Labels constraint-solving and problem-of-the-day are referenced in the workflow frontmatter but don't exist in the repository. The create_discussion itself succeeds, but labels are not applied.
Impact: LOW — Non-fatal warning. Discussion is created successfully; only labels are missing.
Root Cause Analysis
Detection Job Issues (Blocking Safe Outputs)
The EP024 escalation is the most systemic concern today. Three Copilot detection runs failed with authentication errors — different request IDs in each case, suggesting intermittent token/credential issues rather than a systematic misconfiguration. One run (Smoke Gemini) shows a streaming truncation pattern where the detection model starts producing output but the response ends before completing the JSON. This is likely a streaming timeout or network interruption.
The detection failures propagate as a blocker: safe_outputs.if evaluates needs.detection.result == 'success', so any detection failure silently skips all safe outputs, discarding valid agent work.
Safe Output Handler Issues
The single safe output failure (EP019b) is a correct enforcement of security policy — the agent wrote a file to .github/smoke-test-push-{run_id}, which is a protected path. However, the cascade behavior (cancelling subsequent create_issue) is overly aggressive for a smoke test workflow where safe output writes are expected to be low-risk.
Infrastructure Issues (EP025)
The actions/setup not-found errors appear in diverse job types, suggesting that after certain checkout operations (like checking out a memory branch or performing sparse checkout), the main repository's actions/ directory is no longer available. This is a post-checkout state contamination bug.
Recommendations
Critical Issues (Immediate Action Required)
EP024: Copilot Detection Authentication Failures
Priority: High
Root Cause: Intermittent Copilot token auth failures during detection job; streaming truncation in at least 1 case
Recommended Action:
Add retry logic (1-2 retries with backoff) in the detection step when authentication fails
For streaming truncation: add a post-processing check that validates the THREAT_DETECTION_RESULT line is complete before considering the detection done
Monitor: If auth failures persist, investigate whether the Copilot token needs rotation
EP019b: Smoke Claude Agent Writing to .github/ Path
Priority: Medium
Location: Smoke Claude workflow prompt / push_to_pull_request_branch config
Problem: Agent patches target .github/smoke-test-push-{run_id}, a protected prefix
Fix:
Option A: Configure protected-files: fallback-to-issue in smoke-claude's push_to_pull_request_branch handler config so a review issue is created instead of hard failure
Option B: Instruct the smoke test agent to write artifacts to non-protected paths (/tmp/ or smoke-tests/)
Problem: Post-checkout state loses the actions folder, breaking local action references
Fix: Add a Checkout actions folder step before any local action invocation in push_repo_memory and upload_assets jobs (similar to how the safe_outputs job does it)
Affected: 5+ workflows, escalating from 2 to 5 occurrences in 2 days
Configuration Changes
EP_LABELS_MISSING: Create Missing Labels in Repository
Priority: Low
Current: Labels constraint-solving and problem-of-the-day do not exist in github/gh-aw
Recommended: Create these two labels in the repository, or remove them from Constraint Solving workflow frontmatter
Reason: 4th consecutive day of this warning; simple fix
Work Item Plans
Work Item 1: Detection Retry Logic for Authentication Failures
Type: Bug Fix
Priority: High
Description: The Copilot detection job fails with intermittent authentication errors, causing valid agent safe outputs to be silently discarded. A retry mechanism would make detection more robust.
Streaming truncation detected and handled (re-run detection if output is incomplete)
Detection failures log actionable error messages with retry count
Technical Approach: Modify the detection entrypoint to wrap the model call with retry logic; add output validation before setting detection_success=true
Estimated Effort: Medium
Dependencies: Access to detection job entrypoint code
Work Item 2: Fix EP025 actions/setup Availability in push_repo_memory and upload_assets
Type: Bug Fix
Priority: Medium
Description: Various jobs fail because actions/setup is not available after branch checkout operations corrupt the working directory state.
Acceptance Criteria:
push_repo_memory job reliably finds actions/setup
upload_assets job reliably finds actions/setup
No more Can't find 'action.yml' errors in these jobs
Technical Approach: Add a Checkout actions folder step (sparse checkout of the actions/ directory) before any local action reference in affected jobs
Estimated Effort: Small
Dependencies: Identify all jobs affected by this pattern
Work Item 3: Fix Smoke Claude Agent Path for push_to_pull_request_branch (EP019b)
Type: Configuration Fix
Priority: Medium
Description: Smoke Claude creates patches targeting .github/ paths, triggering the protected-file block and cascading failure.
Acceptance Criteria:
Smoke Claude smoke test writes do not target .github/ protected prefix
If a protected-path push is attempted, fallback creates a review issue instead of hard failure
Subsequent safe output messages are not cancelled
Technical Approach: Add protected-files: fallback-to-issue to Smoke Claude's push_to_pull_request_branch config; update agent instructions to use non-protected paths for smoke test artifacts
Estimated Effort: Small
Historical Context
Recent Trend (Last 7 Days)
Date
Success Rate
Hard Failures
Warnings
Main Pattern
2026-03-22
N/A
2
0
EP006 + EP022 candidate
2026-03-23
100%
0
0
Clean run
2026-03-24
97.0%
1
0
EP022: push target branch missing
2026-03-25
100%
0
1
EP_LABELS_MISSING (2nd day)
2026-03-26
N/A
0
0
EP024 first observed (detection), EP025 first observed
Trend: Success rate stable at 95-100% for safe output operations. However, detection failures (EP024) are an escalating pattern that silently discards agent work — first 4 cases on 2026-03-26, then 4 more today.
EP025 escalation: Went from 2 occurrences (2026-03-26) to 5 occurrences today, now affecting 5+ workflows.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
Uh oh!
There was an error while loading. Please reload this page.
-
Executive Summary
Overall system health is good. The single hard safe output failure (EP019 variant in Smoke Claude) is a known pattern related to protected path enforcement. The 4 detection failures blocking safe outputs are a more significant concern—they silently discard valid agent work.
Safe Output Job Statistics
¹
create_issuewas cancelled due to the precedingpush_to_pull_request_branchfailure.Runs that executed safe outputs successfully: Issue Monster (×2), Daily Firewall, PR Triage Agent, Constraint Solving, Weekly Editors Health Check, Smoke Copilot, Smoke Claude (partial)
Runs where safe outputs were SKIPPED due to detection failure: Smoke Create Cross-Repo PR, Smoke Update Cross-Repo PR, Auto-Triage Issues, Smoke Gemini
Error Clusters
Cluster 1: Copilot Detection Authentication Failures (EP024 — HIGH)
detection(blockssafe_outputs)delta:truemarker indicates the final message was a streaming delta that was cut off mid-JSON.Cluster 2: push_to_pull_request_branch Protected Path Violation (EP019b — NEW)
safe_outputs(push_to_pull_request_branch handler).github/smoke-test-push-{run_id}, which falls under the protected path prefix.github/. The handler correctly blocks this, but causes cascade cancellation of the followingcreate_issuemessage.Cluster 3: EP025 actions/setup Not Found — Infrastructure (Multiple Jobs)
push_repo_memory,upload_assets,agent(post-cleanup)actions/setuplocal action is referenced before the repository is fully checked out in certain job configurations. Affectspush_repo_memory,upload_assets, and agent post-cleanup steps.failureeven when the safe_outputs job itself ran correctly (e.g., PR Triage Agent: safe output noop succeeded, but workflow failed due to push_repo_memory). Also contributes to the detection failures seen in Cluster 1.Cluster 4: EP_LABELS_MISSING — Constraint Solving Labels (4th Consecutive Day)
safe_outputs(create_discussion labels step)constraint-solvingandproblem-of-the-dayare referenced in the workflow frontmatter but don't exist in the repository. Thecreate_discussionitself succeeds, but labels are not applied.Root Cause Analysis
Detection Job Issues (Blocking Safe Outputs)
The EP024 escalation is the most systemic concern today. Three Copilot detection runs failed with authentication errors — different request IDs in each case, suggesting intermittent token/credential issues rather than a systematic misconfiguration. One run (Smoke Gemini) shows a streaming truncation pattern where the detection model starts producing output but the response ends before completing the JSON. This is likely a streaming timeout or network interruption.
The detection failures propagate as a blocker:
safe_outputs.ifevaluatesneeds.detection.result == 'success', so any detection failure silently skips all safe outputs, discarding valid agent work.Safe Output Handler Issues
The single safe output failure (EP019b) is a correct enforcement of security policy — the agent wrote a file to
.github/smoke-test-push-{run_id}, which is a protected path. However, the cascade behavior (cancelling subsequentcreate_issue) is overly aggressive for a smoke test workflow where safe output writes are expected to be low-risk.Infrastructure Issues (EP025)
The
actions/setupnot-found errors appear in diverse job types, suggesting that after certain checkout operations (like checking out a memory branch or performing sparse checkout), the main repository'sactions/directory is no longer available. This is a post-checkout state contamination bug.Recommendations
Critical Issues (Immediate Action Required)
THREAT_DETECTION_RESULTline is complete before considering the detection doneBug Fixes Required
EP019b: Smoke Claude Agent Writing to .github/ Path
.github/smoke-test-push-{run_id}, a protected prefixprotected-files: fallback-to-issuein smoke-claude'spush_to_pull_request_branchhandler config so a review issue is created instead of hard failure/tmp/orsmoke-tests/)EP025: actions/setup Not Found After Branch Checkout
push_repo_memory,upload_assetsjob definitionsCheckout actions folderstep before any local action invocation inpush_repo_memoryandupload_assetsjobs (similar to how thesafe_outputsjob does it)Configuration Changes
constraint-solvingandproblem-of-the-daydo not exist in github/gh-awWork Item Plans
Work Item 1: Detection Retry Logic for Authentication Failures
detection_success=trueWork Item 2: Fix EP025 actions/setup Availability in push_repo_memory and upload_assets
actions/setupis not available after branch checkout operations corrupt the working directory state.push_repo_memoryjob reliably findsactions/setupupload_assetsjob reliably findsactions/setupCan't find 'action.yml'errors in these jobsCheckout actions folderstep (sparse checkout of theactions/directory) before any local action reference in affected jobsWork Item 3: Fix Smoke Claude Agent Path for push_to_pull_request_branch (EP019b)
.github/paths, triggering the protected-file block and cascading failure..github/protected prefixprotected-files: fallback-to-issueto Smoke Claude'spush_to_pull_request_branchconfig; update agent instructions to use non-protected paths for smoke test artifactsHistorical Context
Recent Trend (Last 7 Days)
Trend: Success rate stable at 95-100% for safe output operations. However, detection failures (EP024) are an escalating pattern that silently discards agent work — first 4 cases on 2026-03-26, then 4 more today.
EP025 escalation: Went from 2 occurrences (2026-03-26) to 5 occurrences today, now affecting 5+ workflows.
Metrics and KPIs
Next Steps
Checkout actions folderstep topush_repo_memoryandupload_assetsjobsprotected-files: fallback-to-issuein smoke-claude workflowconstraint-solvingandproblem-of-the-dayin repositoryReferences:
Beta Was this translation helpful? Give feedback.
All reactions