[safe-output-health] Safe Output Health Report - 2026-04-11

[github-actions[bot]]

Executive Summary

• Period: 2026-04-11 (last 24 hours)
• Runs Analyzed: 57 completed + 3 in-progress
• Workflows Active: ~35
• Safe Output Items (all runs): 74 across 34 runs
• Safe Output Messages (accessible logs): 25 attempted, 24 succeeded, 1 failed
• Hard Failures: 1
• Error Clusters Identified: 1 (recurring)
• Overall Health: ⚠️ Good — 1 recurring medium-severity failure in Smoke Copilot upload_artifact

---

Safe Output Job Statistics (accessible logs)

Job Type	Executions	Failures	Success Rate
create_issue	2	0	100%
create_discussion	3	0	100%
add_comment	3	0	100%
add_labels	2	0	100%
push_to_pull_request_branch	1	0	100%
submit_pull_request_review	1	0	100%
create_pull_request_review_comment	2	0	100%
update_pull_request	1	0	100%
add_reviewer	1	0	100%
create_code_scanning_alert	1	0	100%
dispatch_workflow	1	0	100%
set_issue_type	1	0	100%
upload_artifact	1	1	0%
noop	1	0	100%
Total	25	1	96%

---

Error Clusters

Cluster 1: upload_artifact — Staging Path Missing (Recurring, Day 2)

• Count: 1 occurrence today (4 total since 2026-04-10)
• Affected Job Type: upload_artifact
• Affected Workflow: Smoke Copilot — §24282845232
• Error:

  ##[error]✗ Message 4 (upload_artifact) failed:
  ERR_VALIDATION: upload_artifact: path does not exist in staging directory: gh-aw
  

• Root Cause: The Copilot agent generates an upload_artifact call referencing path gh-aw (a binary), but this file was never copied to the /tmp/gh-aw/safeoutputs/upload-artifacts/ staging directory before the MCP tool call. The staging artifact safe-outputs-upload-artifacts doesn't exist or doesn't contain gh-aw.
• Impact: Smoke Copilot run marked as failure (conclusion). 8 of 9 other messages succeed. Medium severity.
• Status: ⚠️ Recurring — first seen 2026-04-10, now day 2.

---

Root Cause Analysis

Upload Artifact Staging Issue

The upload_artifact safe-output handler validates that:

1. A staging artifact named safe-outputs-upload-artifacts must be pre-downloaded before the job runs.
2. The path specified by the agent (e.g., gh-aw) must exist within the staging directory.

In both Smoke Copilot runs (today and yesterday), the Copilot agent requests an upload of the gh-aw binary — but the staging artifact is either absent or doesn't include the requested path. This is likely a test scenario where the smoke test agent is intentionally trying to exercise upload_artifact, but the test setup doesn't pre-stage the expected file.

Two possible root causes:

1. The smoke test workflow is missing a step to pre-stage gh-aw to /tmp/gh-aw/safeoutputs/upload-artifacts/.
2. The Copilot agent is generating a path that doesn't match the actual staged file name.

The safe output handler correctly catches and reports this as a validation error rather than silently failing — so the error-detection mechanism works correctly.

Cross-Repo Credential Failures (Agent-Level, Out of Scope)

Two runs had agent-level failures that prevented safe outputs from executing:

• §24282925362 — Smoke Update Cross-Repo PR
• §24283030256 — Smoke Create Cross-Repo PR

Both failed with Bad credentials when checking out githubnext/gh-aw-side-repo. This is a recurring infrastructure issue (day 6+) unrelated to safe output job health.

---

Notable Successes

• Smoke Claude (§24282404926): All 10 messages succeeded, including a full PR review pipeline (create_issue, create_code_scanning_alert, add_comment, update_pull_request, create_pull_request_review_comment ×2, add_reviewer, submit_pull_request_review, push_to_pull_request_branch).
• Auto-Triage Issues (§24282706292): add_labels and create_discussion succeeded.
• Weekly Editors Health Check (§24283057325): Agent correctly emitted noop due to network restrictions — the safe-output workflow gracefully handled this.

---

Recommendations

Critical Issues (Immediate Action Required)

None today.

Medium Priority: upload_artifact Smoke Test Setup

1. Issue: Smoke Copilot's upload_artifact test fails because the agent-requested path gh-aw doesn't exist in the staging directory.
   • Priority: Medium
   • Root Cause: Missing pre-staging step in smoke test setup, or agent generates incorrect path
   • Recommended Action: Review the Smoke Copilot workflow — either add a step that copies the gh-aw binary to /tmp/gh-aw/safeoutputs/upload-artifacts/gh-aw before the agent runs, or update the smoke test scenario to use a fixture file instead.
   • Affected: Smoke Copilot (upload_artifact validation)

Work Item: Fix Smoke Copilot upload_artifact Test Setup

• Type: Bug Fix
• Priority: Medium
• Description: The Smoke Copilot workflow attempts to exercise upload_artifact by uploading a gh-aw binary, but the pre-staging step is missing or not configured correctly. The agent calls upload_artifact with path: "gh-aw" but the staging area never contains that file.
• Acceptance Criteria:
  • upload_artifact in Smoke Copilot completes successfully
  • Smoke Copilot workflow concludes with success
• Technical Approach: Add a workflow step to create a fixture file at /tmp/gh-aw/safeoutputs/upload-artifacts/gh-aw (e.g., a dummy binary or test file), OR modify the Copilot smoke test prompt to use a different, simpler upload path.
• Estimated Effort: Small

---

Historical Context

14-day trend: safe output failure counts

Date	Runs	Failures	Health
2026-03-29	42	4	⚠️
2026-03-30	127	5	⚠️
2026-03-31	67	1	✅
2026-04-01	100	2	✅
2026-04-02	177	10	❌ (7 from API rate limit burst)
2026-04-03	24	0	✅
2026-04-04	46	0	✅
2026-04-05	198	0	✅
2026-04-06	—	0	✅
2026-04-07	—	1	⚠️
2026-04-08	50	0	✅
2026-04-09	74	0	✅ (MCP gateway issue affected agents, not safe outputs)
2026-04-10	299	4	⚠️ (upload_artifact×2, update_issue context, resolve thread perm)
2026-04-11	57	1	⚠️ (upload_artifact staging, day 2)

Active Recurring Patterns

Pattern	First Seen	Last Seen	Status
upload_artifact_staging_missing	2026-04-10	2026-04-11	🔴 Active (day 2)
cross_repo_bad_credentials	2026-04-06	2026-04-11	🔴 Active (day 6+, agent-level)
invalid_event_context	2026-03-29	2026-04-10	🟡 Recurring
resolve_review_thread_permission	2026-04-10	2026-04-10	🟡 Monitor

Metrics and KPIs

• Overall Safe Output Success Rate (accessible runs): 96.0% (24/25 messages)
• Most Reliable Job Types: create_discussion, create_issue, add_comment, add_labels — 100%
• Most Problematic Job Type: upload_artifact — 0% (1/1 failed today)
• Trend: Improving after yesterday's 4-failure day; upload_artifact staging issue persists

Next Steps

• Fix Smoke Copilot upload_artifact staging setup (medium priority, ongoing failure)
• Investigate expired/missing PAT for githubnext/gh-aw-side-repo (cross-repo credential issue, day 6+)
• Monitor resolve_pull_request_review_thread permission error (new in 2026-04-10, not seen today)

References:

• §24282845232 — Smoke Copilot (upload_artifact failure)
• §24282404926 — Smoke Claude (all 10 messages success)
• §24283137232 — This audit run

Generated by Safe Output Health Monitor · ● 365.1K · ◷

• expires on Apr 12, 2026, 1:17 PM UTC

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-04-12T13:17:20.658Z.

Closed by Workflow