[safe-output-health] Safe Output Health Monitor - 2026-09-06

[github-actions[bot]]

Safe Output Health Monitor — Daily Report (2026-09-06)

Executive Summary

This audit covers all agentic workflow runs in github/gh-aw from the ~24-hour window ending 2026-09-06, focused exclusively on safe-output job executions (the unified safe_outputs GitHub Actions job, which processes every safe-output type — create_issue, create_discussion, add_comment, approve_workflow_run, create_pull_request, etc. — inside a single Process Safe Outputs step). Agent-job and detection-job failures are explicitly out of scope for this monitor.

Of 257 workflow runs analyzed (spanning ~90 distinct workflows), 233 executed a safe_outputs job. Of those, 232 succeeded and 1 failed, a 99.57% success rate — a strong rebound from yesterday's 97.4% (7 failures / 275 runs). The single failure is very likely the 12th occurrence of the long-standing, already-root-caused PR Sous Chef approve_workflow_run protected-files-decline-miscategorized pattern, first confirmed on 2026-08-26 and still unresolved. No new failure signatures were observed, and none of the other actively-tracked recurring patterns recurred in this window.

Safe Output Job Statistics

Note on methodology: github/gh-aw implements all safe-output types inside one unified job named safe_outputs (step Process Safe Outputs, followed by Upload Safe Outputs Items) — there are no separate per-safe-output-type jobs. The table below therefore reports job-level statistics, with the specific safe-output tool(s) invoked in the one failing run called out separately.

Metric	Value
Runs analyzed	257
Distinct workflows active	~90
safe_outputs job executions	233
safe_outputs job successes	232
safe_outputs job failures	1
Success rate (of executed)	99.57%
Yesterday's success rate (2026-09-05)	97.4%

Failing run detail:

Field	Value
Run ID	33956577324
Workflow	PR Sous Chef (.github/workflows/pr-sous-chef.lock.yml)
Trigger	schedule
Engine	Pi (openai/gpt-5.4)
Failing step	Process Safe Outputs (job safe_outputs, conclusion failure)
Safe-output tool calls in batch	approve_workflow_run x2, create_issue x1
Tool-success-rate grader	1.0 (passed, threshold ≥0.8)
Tool-failure-count grader	0 (passed, threshold ≤5)
Failure classification	agent_logic

Error Clusters

Cluster 1: PR Sous Chef / approve_workflow_run protected-files-decline-miscategorized (12th occurrence)

• Run: 33956577324
• Category: logic/validation, miscategorized as a hard failure
• Confidence: medium — matched by shape, not raw error text (see caveat below)

All 13 quality graders for this run passed, including tool-success-rate=1.0 and tool-failure-count=0 — meaning none of the 3 safe-output tool calls (approve_workflow_run x2, create_issue x1) was individually flagged as erroring. Yet the safe_outputs job's overall conclusion is failure, and the run is classified failure_kind: agent_logic. This shape — a job-level failure with zero tool-call-level failures — matches the already-confirmed approve-workflow-run-protected-files-decline-miscategorized pattern (open since 2026-08-26, 11 prior occurrences) far more closely than the sibling approve-workflow-run-fork-pr-permission-gap pattern, which throws an explicit Resource not accessible by personal access token API error that would register as a tool failure.

Caveat: the exact protected-file list or decline message could not be retrieved this audit. Prior audits successfully pulled a safe-output-errors.json / workflow-logs/*_safe_outputs.txt artifact via agenticworkflows logs ... artifacts:["all"]; the same call this session returned only mcp_tool_usage call-count metadata (all calls status: "unknown"), with no raw console-log text file. This is recorded as a medium-confidence match, not a confirmed 12th occurrence.

No other tracked recurring signature (Design Decision Gate allowed-files decline, hardcoded Smoke-Copilot discussion temp-id, Smoke Project/Smoke Issues credential failures, submit_pull_request_review non-PR-context, concurrent same-PR write race) recurred in this window.

Root Cause Analysis

Category: Permission/Policy-Classification (recurring, unresolved). The dominant and now sole observed safe-output failure mode continues to be PR Sous Chef's approve_workflow_run tool incorrectly surfacing a correct policy decline (the target PR touches protected files, so auto-approval is intentionally refused) as a hard E099-style job failure, rather than a soft "declined by policy" outcome. This is not a bug in the approval logic itself — the decision to decline is correct — but a classification bug in how the safe-outputs processor reports declines vs. genuine errors. The sibling issue, a genuine PAT/token permission gap for approving fork-PR workflow runs (Resource not accessible by personal access token), is a separate, still-unpatched credential-scope bug tracked under the same tool but a distinct pattern ID (6 occurrences, none today).

Recommendations

Critical (ship now — overdue 10+ days):

1. reclassify-protected-file-decline-as-non-failure — change the safe-outputs processor so approve_workflow_run (and, per the generalized version of this fix, push_to_pull_request_branch's allowed-files check) reports policy declines as a distinct "declined by policy" outcome rather than folding them into the same failure path as genuine errors. This single fix would have eliminated essentially all of PR Sous Chef's tracked failures over the past two weeks (12 of ~15 total occurrences across both variants of this shape).
2. grant-actions-write-fork-pr-scope — confirm and grant the PAT scope needed for the "Approve a workflow run for a fork pull request" REST endpoint, or explicitly document fork-PR approval as an accepted limitation and suppress it from failure metrics.

Bug Fixes:

• None new this audit. Previously identified bugs (create-discussion-createDiscussion-undefined-typeerror, smoke-workflow-hardcoded-discussion-temp-id) had zero occurrences today and remain as tracked in error-patterns.json.

Configuration Changes:

• None required based on today's evidence.

Process Improvements:

• Improve raw-log retrievability for Process Safe Outputs failures: this audit (like the 2026-09-03 and 2026-09-05 audits before it) could not retrieve a safe-output-errors.json / *_safe_outputs.txt artifact for the one failing run despite requesting artifacts:["all"]. Investigate why this succeeds for some runs/workflows but not others, since sub-cause disambiguation for approve_workflow_run failures currently depends entirely on this artifact being present.

Work Item Plans

Priority	Item	Owner action	Status
P0	Ship reclassify-protected-file-decline-as-non-failure	Change safe-outputs processor decline handling	Proposed 2026-08-26, unshipped, 12+ occurrences
P0	Ship grant-actions-write-fork-pr-scope	Grant/confirm PAT scope for fork-PR approval endpoint	Proposed 2026-08-26, unshipped, 6 occurrences
P2	Investigate raw safe-output-errors.json retrieval gaps	Confirm artifact upload conditions for Process Safe Outputs failures	New follow-up, recurring evidence gap (2026-09-03, 2026-09-05, 2026-09-06)

Historical Context / Trends

Success-rate trend since tracking began (2026-08-22 → 2026-09-06): 99.45% → 98.94% → 98.92% → 99.34% → 98.6% → 98.3% → 99.66% (quietest) → 99.52% → 98.84% → 98.95% → 100.0% (first fully clean day, 2026-09-04) → 97.4% (streak broken, 2026-09-05) → 99.57% (today, strong rebound). Today's single failure keeps PR Sous Chef's protected-files-decline pattern as the single most consistently recurring safe-output issue in the repository's history, now spanning 4 separate audit days (2026-08-26, 2026-09-02, 2026-09-06, plus the generalized allowed-files variant on Design Decision Gate across 5 more days) with both proposed fixes still unshipped.

Metrics/KPIs

• Safe-output job success rate: 99.57% (today) vs 97.4% (yesterday) vs 100% (2026-09-04 best day)
• Distinct recurring failure signatures actively tracked: 10
• Recurring signatures with a confirmed root cause but no shipped fix: 4 (protected-files decline, allowed-files decline, fork-PR permission gap, submit_pull_request_review non-PR-context)
• Recurring signatures still gathering data: 4 (not-associated-with-PR, concurrent-PR race, Smoke Project bad credentials, Smoke Issues Jira/Linear credentials)

Next Steps

1. Escalate the two P0 fixes (reclassify-protected-file-decline-as-non-failure, grant-actions-write-fork-pr-scope) — both are now 10+ days old with double-digit occurrence counts and require no further data-gathering, only implementation.
2. Follow up on why raw safe-output-errors.json artifacts were unavailable for today's failure, to restore sub-cause confidence for future audits.
3. Continue passive monitoring of the 4 still-gathering-data signatures; no action needed until they recur.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 🔒 Safe Output Health Monitor · claude · agent · 396.7 AIC · ⌖ 9.32 AIC · ⊞ 10.8K · ◷

• expires on Sep 6, 2026, 8:54 PM UTC-08:00