[safe-output-health] Safe Output Health Audit — 2026-08-28

[github-actions[bot]]

Executive Summary

Audited 286 safe_outputs job executions across the trailing 24h window (~2026-08-27T12:36Z – 2026-08-28T03:21Z). 4 failures, a 98.60% success rate — in line with the last several audits (99.34% on 08-26, 98.92% on 08-25).

Two clusters this cycle:

1. Recurred & root-caused: a previously-dormant push_to_pull_request_branch failure pattern in Design Decision Gate reappeared, and is now confirmed to be a policy guardrail (allowed-files whitelist) correctly declining a bundle — miscategorized as a hard failure rather than an intentional decline.
2. New: three independent workflows (Agent Container Smoke Test, Changeset Generator, Smoke Pi) all failed their safe_outputs jobs within a 23-second window while concurrently writing to the same PR (Make engine domain sets explicit opt-in #56447) and commit — pointing to a same-PR write race rather than 3 unrelated bugs.

The previously-confirmed PR Sous Chef approve_workflow_run permission-gap pattern had zero new occurrences for a 4th consecutive audit.

Safe Output Job Statistics

Metric	Value
safe_outputs job executions analyzed	286
Failed	4
Succeeded	282
Success rate	98.60%
Distinct workflows affected	4 (Design Decision Gate, Agent Container Smoke Test, Changeset Generator, Smoke Pi)
Distinct safe-output types affected	4 (push_to_pull_request_branch, add_comment, update_pull_request, create_issue)

Error Clusters

Cluster	Workflow(s)	Safe-output type(s)	Failed items	Run(s)
Allowed-files policy decline	Design Decision Gate 🏗️	push_to_pull_request_branch	1	§33074418588
Same-PR concurrent write race (PR #56447)	Agent Container Smoke Test, Changeset Generator, Smoke Pi	add_comment, push_to_pull_request_branch, update_pull_request, create_issue	5 (across 3 runs)	§33138018899, §33138018891, §33138018918

Full per-run error detail

§33074418588 — Design Decision Gate 🏗️ (2026-08-27T13:11:35Z–13:13:23Z)

• push_to_pull_request_branch: "Cannot push to pull request branch: bundle modifies files outside the allowed-files list (docs/public/editor/autocomplete-data.json, docs/src/content/docs/reference/frontmatter-full.md, docs/src/content/docs/reference/frontmatter.md, pkg/parser/schema_test.go, pkg/parser/schemas/main_workflow_schema.json, pkg/workflow/role_checks.go, pkg/workflow/role_checks_test.go). Add the files to the allowed-files configuration field or remove them from the bundle."

§33138018899 — Agent Container Smoke Test (2026-08-28T03:15:02Z–03:15:27Z, PR #56447, sha ee852f1)

• add_comment: "Handler returned success: false" (no further detail available)

§33138018891 — Changeset Generator (2026-08-28T03:14:34Z–03:15:51Z, PR #56447, sha ee852f1)

• push_to_pull_request_branch: "Failed to push changes: The process '/usr/bin/git' failed with exit code 1"
• update_pull_request: "ERR_API: update pull request Make engine domain sets explicit opt-in #56447 failed (attempt 1); Retryable: false" (empty original-error body)

§33138018918 — Smoke Pi (2026-08-28T03:14:31Z–03:15:23Z, PR #56447, sha ee852f1)

• create_issue: "ERR_API: create_issue in github/gh-aw failed (attempt 1); Original error: Server Error; Retryable: false"
• add_comment: "Handler returned success: false"

Root Cause Analysis

1. Allowed-files policy decline (Design Decision Gate) — CONFIRMED
push_to_pull_request_branch is working as designed: it refuses to push a bundle that touches files outside the workflow's configured allowed-files whitelist. The failure text names the 7 out-of-scope files explicitly. This is the same shape of problem as the 2026-08-26-confirmed approve_workflow_run protected-files decline: a correct policy rejection reported through the same hard-failure (E099) path as genuine errors. This also root-causes the previously-dormant process-safe-outputs-cross-workflow-2026-08-25 pattern for its Design Decision Gate occurrence — the 2026-08-25 run used an identical safe-output config (push_to_pull_request_branch). The companion 2026-08-25 Designer Drift Audit occurrence (a create_issue-only config) is not explained by this cause and remains open.

2. Same-PR concurrent write race (PR #56447) — NEW, hypothesis, not yet confirmed
Three independent, separately-triggered workflows (Agent Container Smoke Test, Changeset Generator, Smoke Pi) all ran against the same pull_request event — same head commit ee852f170a5e84774fbb12ef74091efc29240948, branch copilot/update-compiler-domain-set-handling, PR #56447 ("Make engine domain sets explicit opt-in") — and all three safe_outputs jobs failed within a 23-second window (03:15:18Z–03:15:41Z).

An initial hypothesis — that the PR closed/merged mid-flight, invalidating in-progress writes — was checked and refuted: PR #56447 was not merged until 03:32:04Z, 17 minutes after these failures, so the PR was still open and actively being written to when all three jobs failed. The PR's own comment thread additionally shows a 4th workflow (Smoke Claude, run 33138018885) writing to the same PR in the same window, confirming at least 4 concurrent writers.

Working hypothesis: concurrent safe_outputs jobs from independently-triggered workflows raced to write to the same branch/PR — simultaneous git push attempts collided (non-fast-forward, exit code 1), and simultaneous PR mutations (comment, body update) produced transient-looking API 500s or silently-failing handlers. Not yet confirmed; needs 1–2 more same-signature recurrences.

3. Opaque add_comment failures — evidence gap
Two failures (Agent Container Smoke Test, Smoke Pi) report only "Handler returned success: false" with no HTTP status or underlying error, preventing a definitive root cause for those specific items.

Recommendations

1. Generalize policy-decline reclassification (extends a 2026-08-26 recommendation) — any safe-output tool that declines due to a configured whitelist/policy check (protected-files, allowed-files, etc.) should report a distinct "declined by policy" outcome, not the same E099 hard-failure path as genuine errors. Currently affects both approve_workflow_run (protected-files) and push_to_pull_request_branch (allowed-files).
2. Improve add_comment error propagation — surface the underlying HTTP status/error body instead of a bare Handler returned success: false, so future audits aren't blocked by this evidence gap.
3. Consider a concurrency guard for same-PR writes — a per-PR lock, or retry-with-backoff on non-fast-forward push rejections and transient update_pull_request errors, for push_to_pull_request_branch / update_pull_request. Low urgency pending more data points.
4. No action needed on the PR Sous Chef approve_workflow_run PAT permission gap this cycle (still open, zero new occurrences, tracked separately).

Work Item Plans

• WI-1: Generalize policy-decline classification across all safe-output types (protected-files + allowed-files checks) so declines don't count as job failures. Owner: safe-outputs processor maintainers.
• WI-2: Add structured error detail (status code / API error body) to the add_comment handler's failure path.
• WI-3: Investigate whether push_to_pull_request_branch / update_pull_request should retry-with-backoff on concurrent-write conflicts before surfacing failure; monitor for further same-PR clustering before committing to a fix.

Historical Context / Trends

Date	Executions	Failures	Success rate	Primary finding
2026-08-22	841	2	99.45%	PR Sous Chef approve_workflow_run failures (early occurrences)
2026-08-23	224	2	98.94%	PR Sous Chef recurrence + 1 infra flake
2026-08-25	278	3	98.92%	PR Sous Chef recurrence + first cross-workflow sighting (Designer Drift Audit, Design Decision Gate)
2026-08-26	304	2	99.34%	PR Sous Chef root-caused (permission gap + protected-files miscategorization)
2026-08-28	286	4	98.60%	Cross-workflow pattern recurred & root-caused; new same-PR concurrency cluster

Metrics / KPIs

• 5-audit rolling success rate: 99.45% → 98.94% → 98.92% → 99.34% → 98.60% (no sustained degrading or improving trend; within normal noise band).
• Recurring-pattern count: 2 active (PR Sous Chef permission gap — dormant 4 audits; same-PR concurrency race — new, 1 audit).
• 1 pattern closed-out-with-root-cause this cycle (cross-workflow allowed-files pattern).

Next Steps

• Monitor for further same-PR concurrent-write clustering (WI-3 trigger condition: ≥2 more occurrences with the same signature).
• Track whether Smoke Claude (run 33138018885, same PR Make engine domain sets explicit opt-in #56447 window) also failed — not yet checked; would help confirm whether the race affects all concurrent writers or only a subset.
• Re-check PR Sous Chef approve_workflow_run PAT permission gap status — still unresolved but dormant; escalate if it recurs.

References:

• §33074418588 — Design Decision Gate, allowed-files decline
• §33138018891 — Changeset Generator, same-PR race cluster
• §33138018918 — Smoke Pi, same-PR race cluster

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 🔒 Safe Output Health Monitor · claude · agent · 274.2 AIC · ⌖ 10.4 AIC · ⊞ 9K · ◷

• expires on Aug 28, 2026, 11:19 PM UTC-08:00