[safe-output-health] Safe Output Health Monitor — 2026-09-02 Daily Audit

[github-actions[bot]]

Executive Summary

Audited all agentic workflow runs from 2026-09-01T10:33 UTC to 2026-09-02T03:56 UTC (~17.5h window, 290 runs cached locally). Scope is strictly limited to safe_outputs job failures — agent/detection/activation job failures are monitored elsewhere and excluded (e.g. run §33497991476, Daily Go Test Parallelizer, failed only at its agent job and was correctly excluded).

3 safe_outputs job failures out of 259 executions (255 success, 1 skipped) → 98.84% success rate. All 3 failures were partial (per-item isolated) — no run lost 100% of its safe outputs. 2 of 3 are new occurrences of an already-known, still-unresolved recurring pattern; 1 is a brand-new code-level bug.

Safe Output Job Statistics

Metric	Value
Runs analyzed (locally cached)	290
safe_outputs job executions	259
— success	255
— skipped	1
— failure	3
Success rate	98.84%
Distinct workflows affected	2 (PR Sous Chef, Auto-Triage Issues)
Distinct root causes	2 (1 recurring, 1 new)

Error Clusters

Cluster 1 — approve_workflow_run protected-files decline miscategorized (66.7% of today's failures)

• Workflow: PR Sous Chef, runs §33533253304 and §33563671627, ~5h apart
• 7 individual approve_workflow_run items failed across 3 distinct PRs: Add shared Playwright title smoke test #57721 (18 protected files, mostly paired .lock.yml/.md smoke-test workflows + .github/aw/actions-lock.json), Bump charm.land/bubbles/v2 from 2.2.0 to 2.2.1 #57669 (go.mod, go.sum), Apply mention limits to output content instead of allowlists #57747 (~30 .github/workflows/*.lock.yml regenerated files)
• Each run's create_issue item succeeded independently in the same batch (posted as a comment on pre-existing issue [pr-sous-chef] pr-sous-chef run report #57526 via group-by-day dedup), confirming per-item failure isolation continues to hold — one failed item never aborts the batch.

Cluster 2 — create_discussion TypeError (33.3% of today's failures, NEW)

• Workflow: Auto-Triage Issues, run §33508317422
• Error: Cannot read properties of undefined (reading 'createDiscussion')
• add_labels succeeded independently in the same run (partial_success, 1/2 items).

Root Cause Analysis

Cluster 1 (recurring, root cause confirmed 2026-08-26): approve_workflow_run is functioning correctly — it declines to auto-approve a workflow run when the target PR modifies protected files, exactly as configured (protected_files list includes go.mod, go.sum, .github/workflows/** lock files, etc.). The bug is that this correct policy decline is surfaced as a hard job failure (##[error]) instead of a soft skip/neutral outcome, which pollutes success-rate metrics and generates false-alarm noise. This is now the dominant failure mode for PR Sous Chef: 11 total occurrences over 7+ days, with today alone contributing 7 of those across 2 runs and 3 PRs. The proposed fix (reclassify-protected-file-decline-as-non-failure) has not shipped since it was first proposed on 2026-08-28.

Cluster 2 (new): Immediately before the failure, the handler successfully fetched discussion categories (Fetched discussion categories for github/gh-aw, Using category: Audits (name)) — which rules out all three causes the error message itself lists (discussions disabled, invalid category ID, insufficient permissions). This points to a genuine code-level bug: some client or GraphQL-mutation object is undefined at the moment createDiscussion is invoked, despite the preceding category-fetch call succeeding on what should be the same client. Only 1 occurrence so far — needs more data points to determine if it's deterministic or intermittent.

Recommendations

Immediate actions:

1. Re-run/monitor Auto-Triage Issues' next scheduled execution to see if the create_discussion TypeError recurs (would confirm determinism vs. a transient client-init race).

Bug fixes:

1. [High priority, 7+ days open] Ship reclassify-protected-file-decline-as-non-failure for approve_workflow_run (and generalize to push_to_pull_request_branch's allowed-files check, per the related Design Decision Gate pattern) — treat protected/allowed-files policy declines as a soft skip/neutral result, not a job failure.
2. [New] Investigate the create_discussion handler code path for a client/GraphQL-mutation object that can be undefined immediately after a successful category fetch on the same client instance — likely an object lifecycle or scoping bug in the safe-outputs create_discussion implementation.

Config changes: None required — PR Sous Chef's protected_files list is behaving exactly as configured; the issue is purely in failure classification, not configuration.

Process improvements: None beyond the above; per-item failure isolation and group-by-day issue dedup both continue to work as designed.

Work Item Plans

Item	Owner area	Effort	Blocking?
Reclassify protected/allowed-files policy declines as non-failures (approve_workflow_run + push_to_pull_request_branch)	safe-outputs processor (process_safe_outputs.cjs)	Medium	No — cosmetic/metrics-quality, but now the top recurring noise source
Root-cause create_discussion undefined-client TypeError	safe-outputs create_discussion handler	Small–Medium	No — single occurrence so far

Historical Context / Trends

• Success rate has stayed in a tight 98.3%–99.7% band since 2026-08-22 (11 audits); today's 98.84% is within normal range.
• The Design Decision Gate allowed-files pattern (5 occurrences 2026-08-25→2026-08-30) had its 3rd consecutive clean day — 0 new occurrences.
• The approve_workflow_run protected-files-decline pattern, previously quiet since 2026-08-26, had its largest single-day spike yet today (+7 occurrences, 2 runs) — now clearly PR Sous Chef's dominant and most persistent unresolved issue.
• All other previously tracked patterns (fork-PR PAT permission gap, hardcoded smoke-discussion temp-id, approve_workflow_run not-associated-with-PR, same-PR concurrency race, Smoke Claude submit-review non-PR-context, Smoke Project bad-credentials, checkout-actions-folder infra flake) had zero new occurrences today.

Metrics / KPIs

• Safe output job success rate: 98.84% (255/259, +1 skipped)
• Failures fully root-caused: 3/3 (2 recurring + 1 hypothesis-stage new pattern)
• Recurring-pattern share of today's failures: 66.7% (2 of 3 failures, 7 of 8 total failed items)
• Days since top recurring-pattern fix proposed without shipping: 7+ (reclassify-protected-file-decline-as-non-failure, proposed 2026-08-28)

Next Steps

1. Escalate the unshipped reclassify-protected-file-decline-as-non-failure fix — it is now responsible for the clear majority of all safe_outputs job failures across the past week.
2. Continue monitoring Auto-Triage Issues' create_discussion calls for recurrence of the TypeError.
3. No action needed on any other tracked pattern — all remain quiet.

References:

• §33508317422 — Auto-Triage Issues, create_discussion TypeError (new)
• §33533253304 — PR Sous Chef, approve_workflow_run protected-files decline (PRs Add shared Playwright title smoke test #57721, Bump charm.land/bubbles/v2 from 2.2.0 to 2.2.1 #57669)
• §33563671627 — PR Sous Chef, approve_workflow_run protected-files decline (PR Apply mention limits to output content instead of allowlists #57747)

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

• api.anthropic.com

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "api.anthropic.com"

See Network Configuration for more information.

Generated by 🔒 Safe Output Health Monitor · claude · agent · 393.7 AIC · ⌖ 9.07 AIC · ⊞ 10.8K · ◷

• expires on Sep 2, 2026, 8:52 PM UTC-08:00