[safe-output-health] Safe Output Health Monitor — 2026-08-23 Daily Report

[github-actions[bot]]

Executive Summary

• Runs analyzed (last 24h): 224
• Distinct workflows active: 72
• Safe output jobs executed: 189 (of 224 total; 35 skipped because no safe-output was declared/triggered for those runs)
• Safe output jobs failed: 2
• Success rate (executed jobs): 98.9 percent
• Error clusters identified: 2 -- both in the same workflow, PR Sous Chef (schedule-triggered, main branch)

This repo compiles safe-output handling into a single consolidated safe_outputs job per run (not per-output-type jobs), which processes every declared safe-output type via one Process Safe Outputs step. The statistics and clusters below are reported against that architecture.

One of the two clusters is a confirmed recurring failure -- this is its 3rd occurrence across the last two daily audits (2026-08-22 and 2026-08-23), always in PR Sous Chef, always failing the same Process Safe Outputs step.

Safe Output Job Statistics

Metric	Count
Total runs analyzed	224
safe_outputs job present	224
safe_outputs job skipped (no safe-output declared this run)	35
safe_outputs job executed	189
safe_outputs job succeeded	187
safe_outputs job failed	2
Success rate (of executed)	98.9 percent
Runs with non-empty missing_tools/missing_data/mcp_failures	0

Error Clusters

Cluster 1 -- Recurring: Process Safe Outputs step failure (PR Sous Chef)

• Run: https://github.com/github/gh-aw/actions/runs/32606608661 (run 32606608661) -- created 2026-08-22T23:57:39Z, job duration 23.0s, engine pi (copilot/gpt-5.4)
• Failed step: Process Safe Outputs (all preceding steps -- checkout, download artifacts, git config -- succeeded)
• This is the 3rd occurrence of this exact signature (workflow + job + step): also seen in runs 32543132845 and 32526302837 on 2026-08-22.

Cluster 2 -- New: Checkout actions folder failure (PR Sous Chef)

• Run: https://github.com/github/gh-aw/actions/runs/32599858918 (run 32599858918) -- created 2026-08-22T21:31:57Z, job duration 25.0s
• Failed step: Checkout actions folder, the very first substantive step -- everything downstream (Process Safe Outputs included) was skipped as a result.
• Same run's independent agent job also failed, at Execute Pi CLI (out of scope for this monitor, noted for correlation only).
• First occurrence of this specific step failing -- treat as a single data point, not yet a confirmed pattern.

Root Cause Analysis

Cluster 1 (Process Safe Outputs, recurring):
Raw stderr for this step is not retrievable through the agenticworkflows audit/logs tools available to this monitor -- only step-level pass/fail status is exposed, with no console text or artifact capturing the actual validation/API error. Based on the workflow's own prompt content (reviewed in the 2026-08-22 audit), PR Sous Chef declares an unusually large and varied batch of safe-output types per run -- add-comment, approve-workflow-run (with a strict allowlist), resolve-pull-request-review-thread (requires exact PRRT_-prefixed node IDs), dismiss-pull-request-review, update-pull-request, push-to-pull-request-branch, create-issue -- and the prompt contains defensive instructions warning the agent against exactly these validation pitfalls. The working hypothesis is that one malformed item in the batch aborts the entire consolidated Process Safe Outputs step, rather than that item failing in isolation while the rest of the batch proceeds.

Cluster 2 (Checkout actions folder):
Looks like transient runner/checkout infrastructure flakiness rather than a safe-output logic bug -- it failed before any safe-output processing began, and the same run's agent job independently failed at a CLI execution step, consistent with a runner- or network-level issue affecting the whole run. Not confirmed as causal or recurring with only one data point.

Recommendations

Critical / Bug fix:

1. Add per-item error isolation to the Process Safe Outputs step so a single malformed safe-output item (bad ID prefix, allowlist mismatch, etc.) logs an error for that item and continues processing the rest of the batch, instead of failing the whole job. This directly targets Cluster 1, which has now recurred 3 times in 2 days with zero change in signature.
2. Until (1) ships, add stderr/error-detail capture (upload as a build artifact or step summary) for the Process Safe Outputs step specifically -- this monitor currently has no way to see why the step fails beyond pass/fail, which blocks precise root-causing.

Process:
3. Re-audit PR Sous Chef specifically after the next few scheduled runs to confirm whether Cluster 2 (Checkout actions folder) recurs -- if it does, escalate as infra flakiness; if it does not reappear, no action needed.

Work Item Plans

• P1: Isolate safe-output batch processing per item -- owner: gh-aw safe-outputs maintainers. Scope: modify the Process Safe Outputs step logic so one item's validation/API failure does not abort the whole batch. Acceptance: a run with 1 malformed plus N valid safe-output items completes with N items applied and 1 item reported as a per-item failure, job conclusion reflects partial success rather than hard failure.
• P2: Add error-detail capture for Process Safe Outputs step -- owner: gh-aw observability/tooling maintainers. Scope: ensure stderr or a structured error object from this step is captured into an artifact the agenticworkflows audit/logs tools can surface. Acceptance: next occurrence of this failure signature yields a specific error message in the audit output, not just step conclusion.

Historical Context

• 2026-08-22 audit (841 runs, roughly 23h window): 2 safe_outputs failures, both PR Sous Chef / Process Safe Outputs, 99.45 percent success rate.
• 2026-08-23 audit (this report, 224 runs, 24h window): 2 safe_outputs failures -- 1 repeat of the same Process Safe Outputs signature, 1 new Checkout actions folder signature, 98.9 percent success rate.
• Trend: the Process Safe Outputs failure in PR Sous Chef is flat and unresolved across both audits -- same workflow, same job, same step, no evidence of a fix landing between windows.

Metrics and KPIs

• Safe-output job success rate (executed): 98.9 percent (187 of 189)
• Safe-output job skip rate: 15.6 percent (35 of 224)
• Recurring failure signatures: 1 (Process Safe Outputs / PR Sous Chef, 3 total occurrences)
• New failure signatures this window: 1 (Checkout actions folder / PR Sous Chef, 1 occurrence)
• Workflows with any safe-output failure this window: 1 of 72 active (PR Sous Chef)

Next Steps

• Track a fix for per-item isolation in Process Safe Outputs (Work Item P1 above)
• Add error-detail capture for this step so future audits can pinpoint the exact validation failure (Work Item P2 above)
• Watch PR Sous Chef's next few scheduled runs for recurrence of either failure signature
• No action needed on the other 187 successful safe-output executions or the 35 correctly-skipped runs

References:

• Run 32606608661: https://github.com/github/gh-aw/actions/runs/32606608661 (recurring Process Safe Outputs failure)
• Run 32599858918: https://github.com/github/gh-aw/actions/runs/32599858918 (new Checkout actions folder failure)
• Run 32543132845: https://github.com/github/gh-aw/actions/runs/32543132845 (prior occurrence of the recurring failure, 2026-08-22 audit)

Generated by 🔒 Safe Output Health Monitor · agent · 245.3 AIC · ⌖ 9.41 AIC · ⊞ 10.8K · ◷

• expires on Aug 23, 2026, 8:47 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-08-24T04:47:58.871Z.

Closed by Workflow