[safe-output-health] Safe Output Health Monitor — 2026-09-22

[github-actions[bot]]

Executive Summary

Audited 227 cached agentic workflow runs from the last 24 hours, focused exclusively on safe output job health (the consolidated safe_outputs job that executes create_discussion, create_issue, add_comment, add_labels, approve_workflow_run, update_pull_request, etc. via its single "Process Safe Outputs" step). 2 of 227 runs (0.88%) had a failing safe_outputs job — a 99.12% job success rate. Both failures show partial success: some items in the batch completed (a discussion was created; two PRs were updated and commented on), while specific item types failed, consistent with this system's well-established per-item-isolation behavior. One of today's two failures (add_labels in Auto-Triage Issues) is a brand-new signature never seen in the 13 prior daily audits on record since 2026-08-22; the other (approve_workflow_run in PR Sous Chef) is the 13th occurrence of a long-standing, already root-caused recurring issue whose fix remains unshipped.

Job Statistics

Metric	Value
Runs analyzed (24h window)	227
Runs with a safe_outputs job failure	2
safe_outputs job success rate	99.12%
Distinct workflows affected	2 (Auto-Triage Issues, PR Sous Chef)
Brand-new failure signatures today	1 (add_labels, Auto-Triage Issues)
Recurring signature re-observed today	1 (approve_workflow_run, PR Sous Chef — 13th occurrence)

Note: the safe_outputs job is a single consolidated job per run (not split per safe-output type); all safe-output writes for a run happen inside its one "Process Safe Outputs" step, and failures are isolated per-item rather than aborting the whole batch.

Error Clusters

Cluster 1 — Auto-Triage Issues / add_labels (NEW, 1 occurrence)

• Run: 35673928865 (Clean up action cache files left behind by tests #3976), engine codex (openai/gpt-5.3-codex), 2026-09-22T00:56:29Z
• Step: "Process Safe Outputs", failed in ~2 seconds
• create_discussion succeeded (discussion #62505 created)
• add_labels (1 call) did not appear in the run's successful-items list — this is what flipped the job to failure
• No previously-tracked signature in 13 prior audits covers an add_labels failure; raw error text was not retrievable this audit.

Cluster 2 — PR Sous Chef / approve_workflow_run (RECURRING, 13th occurrence)

• Run: 35681273619 (GitHub MCP Remote Server Tools Report - December 2025 #5740), engine pi (copilot/gpt-5.4), 2026-09-22T02:55:50Z
• Step: "Process Safe Outputs", failed in ~14 seconds
• update_pull_request x2 and add_comment x2 succeeded (PR #62490, PR #62484)
• approve_workflow_run (6 calls) all failed — matches the long-confirmed recurring signature (fork-PR PAT permission gap, or a protected-files decline miscategorized as a hard failure); exact sub-cause unconfirmed today (raw error text unavailable)
• create_issue (1 call) also did not produce a result — a new data point for this workflow, possibly related to a prior create_discussion handler bug seen 2026-09-02

Root Cause Analysis

• approve_workflow_run (PR Sous Chef): Root-caused since 2026-08-26 across two confirmed sub-causes — (a) the safe_outputs job's PAT lacks permission to approve fork-PR workflow runs ("Resource not accessible by personal access token"), a genuine scope bug; (b) the tool correctly declines to approve PRs touching protected files, but the decline is surfaced via the same hard-failure code path as genuine errors, a miscategorization. 13 occurrences over a month, fix still unshipped.
• add_labels (Auto-Triage Issues): Not yet root-caused — first occurrence. The fact that a sibling create_discussion call in the same batch succeeded confirms per-item isolation is working, but the specific cause of the label failure is unknown pending raw error text.
• create_issue (PR Sous Chef): Not yet root-caused — first occurrence for this workflow. Could be a genuine handler bug (structurally similar to the 2026-09-02 create_discussion "Cannot read properties of undefined" TypeError) or a benign no-op from dedup logic that simply doesn't populate a result URL.
• Recurring data gap: In roughly 5 of the last 8 audits (including today), the raw per-item error text (safe-output-errors.json / workflow-logs/*_safe_outputs.txt) was not retrievable even when the underlying log-fetch call succeeded — it returned only the same structured summary already available locally, not the console text needed to confirm exact error messages. This is now the single biggest blocker to quickly root-causing new failure signatures.

Recommendations

Immediate actions:

1. Ship the already-proposed, long-pending fix: reclassify approve_workflow_run's protected-files decline as a non-failure outcome instead of a hard job failure (proposed 2026-08-26, still open after 13 occurrences).
2. Investigate why the safe_outputs job's PAT lacks fork-PR workflow-run-approval scope, or explicitly document this as a permanent limitation and suppress the resulting error.

New investigations to open:
3. Retrieve raw error text (next occurrence or targeted log pull) for the add_labels failure in Auto-Triage Issues run 35673928865 to determine root cause.
4. Retrieve raw error text for the create_issue failure in PR Sous Chef run 35681273619, and check whether it shares code with the previously-confirmed create_discussion undefined-object bug.

Process improvement:
5. Stabilize raw per-item error-text retrieval for safe_outputs job failures — it currently succeeds inconsistently across audits, which slows down root-cause confirmation for every new signature.

Work Item Plans

Item	Priority	Owner area	Status
Reclassify protected-files/allowed-files declines as non-failures (approve_workflow_run, push_to_pull_request_branch)	High	safe-outputs processor	Proposed 2026-08-26/2026-08-28, unshipped
Grant/confirm fork-PR approval scope for safe_outputs PAT	High	repo/org token config	Proposed 2026-08-26, unshipped
Root-cause add_labels failure (Auto-Triage Issues)	Medium	safe-outputs processor	New today, needs more data
Root-cause create_issue no-result failure (PR Sous Chef)	Medium	safe-outputs processor	New today, needs more data
Improve raw-log retrieval reliability for safe_outputs failures	Medium	tooling/observability	New today

Historical Context

This is the 14th audit in this series (daily since 2026-08-22, with a few gaps). Success rates have ranged 97.4%–100% across that history, with two fully clean days (2026-09-04, 2026-09-11). The dominant recurring issue throughout has been PR Sous Chef's approve_workflow_run (13 total occurrences now), alongside a similar-shaped push_to_pull_request_branch allowed-files decline in Design Decision Gate (6 occurrences, last seen 2026-09-05) — both are policy-correct declines miscategorized as hard failures, and both fixes remain unshipped a month later. Several other lower-frequency signatures remain open with insufficient data (Smoke Project bad-credentials, Smoke Issues Jira/Linear credentials, submit_pull_request_review non-PR-context).

Metrics & KPIs

• Safe output job success rate (today): 99.12% (225/227)
• Safe output job success rate (trailing history): 97.4%–100%, mostly 98–99.7%
• Longest-standing unresolved recurring issue: approve_workflow_run protected-files/permission failures — 13 occurrences, open since 2026-08-22 (31 days)
• New failure signatures introduced today: 2 (add_labels, create_issue)

Next Steps

1. Prioritize shipping the two long-pending reclassification/permission fixes for approve_workflow_run — this remains the single most consistently recurring issue in the monitor's history.
2. Gather a second data point on both new signatures (add_labels, create_issue) before proposing a specific code fix.
3. Investigate the intermittent raw-log-retrieval gap so future audits can root-cause new signatures in a single pass.

---

Automated report generated by the Safe Output Health Monitor. Findings are persisted to /tmp/gh-aw/cache-memory/safe-output-health/ for trend tracking across future audits.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 🔒 Safe Output Health Monitor · claude · agent · 381 AIC · ⌖ 8.45 AIC · ⊞ 11K · ◷

• expires on Sep 22, 2026, 9:00 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-09-23T05:00:09.378Z.

Closed by Workflow