[safe-output-health] 🏥 Safe Output Health Report — 2026-07-24

[github-actions[bot]]

Executive Summary

• Period: Last 24h (window ~2026-07-23T23:58Z → 2026-07-24T05:15Z)
• Runs analyzed: 78 (all run_summary.json job conclusions read directly from job_details[])
• Workflows active: ~25 distinct
• Safe-output jobs executed: 78
• Safe-output jobs failed (in-scope): 2
• In-scope success rate: 97.44% (76 / 78)
• Error clusters: 1 — pr_sous_chef_scheduled_target_resolution (both failures)

Two Process Safe Outputs hard failures, both the same workflow — PR Sous Chef — on scheduled/main runs ~4h apart, each emitting a mixed update_pull_request + add_comment + create_issue payload. Combined with the identical 07-23 failure, this is 3 hard-fails of PR Sous Chef scheduled runs in 2 days. The review-comment family (07-20 / 07-23) did not recur this window.

Safe-Output Job Statistics

Metric	Count
safe_outputs jobs executed	78
success	76
in-scope hard failure	2
out-of-scope run failures (agent/detection) w/ clean handoff	several (all safe_outputs=success)

Handler mix in the 2 failed runs (the only write-actuating safe_outputs jobs observed this window):

Handler	Items across failed runs	Note
update_pull_request	5	target-requiring — implicated
add_comment	6	target-requiring — implicated
create_issue	2	collateral (does not need a PR target)

Metrics caveat: behavior_fingerprint.actuation_style read read_only on all 78 runs (a long-standing undercount). The window was predominantly read-only reviewer/nudge runs; the only write actuations surfaced were the two PR Sous Chef failures.

Error Cluster: pr_sous_chef_scheduled_target_resolution

• Count: 2 occurrences this window (+1 on 07-23 = 3 in 2 days)
• Affected job types: update_pull_request, add_comment
• Affected workflow: PR Sous Chef (schedule, main)
• Root cause (family-inferred): PR Sous Chef is schedule-triggered, so there is no triggering PR/issue context. The update_pull_request / add_comment handlers hard-fail when they cannot resolve a target PR, unlike create_pull_request_review_comment which soft-skips under the same missing-context condition. This soft-skip-vs-hard-fail handler inconsistency is the long-tracked target_star_* family root cause.
• Impact: Medium/production — reds the scheduled run and drops the intended nudge writes; the paired create_issue is collateral.

Failure details (2 runs)

Run	Time	Items	agent	detection	safe_outputs	firewall blocks
§30057263862	00:49Z	update_pull_request×4, add_comment×4, create_issue×1 (9)	success	success	failure	0
§30068020977	04:54Z	update_pull_request×1, add_comment×2, create_issue×1 (4)	success	success	failure	0

Decisive isolation: PR Sous Chef ran 6× this window (all schedule/main). The 4 read-only/noop runs (23:58Z, 01:25Z, 02:29Z, 03:48Z) succeeded; only the 2 write-actuating runs failed. The failure is specific to actuating target-requiring writes on a scheduled run — not infrastructure and not a blanket PR Sous Chef regression.

Observability Gap (recurring, 6th+ occurrence)

The exact Process Safe Outputs step stdout/stderr is not pre-bundled in run artifacts. This window the agenticworkflows logs CLI ran quickly (8.9s, no timeout) but returned 0 runs for the {workflow_name:"PR Sous Chef", start_date:"-1d"} filter, so the step log was still unrecoverable. Both failures therefore have UNCONFIRMED exact error text — root cause is inferred from item type + trigger context + family history. This remains the #1 cross-cutting remediation ask.

Standing-Open Signatures

• review_path_unresolved_422 — NOT recurred this window. Full reviewer suite (Matt Pocock, PR Code Quality, Test Quality Sentinel, Impeccable Skills, Design Decision Gate) present at scale but all read-only → no line-anchored review comments, no 422, pr_review_buffer.cjs:554 Path-variant fix UNVALIDATED (56th audit). The two recent in-prod hard-fails (07-20, 07-23) remain the standing concern for this family.
• changeset_generator_bundle_hardfail — Changeset Generator ABSENT (~28 days since last actuation 2026-06-26). patch-format:BUNDLE bundle-transport signature not re-exercised. Highest-priority UNVALIDATED open signature.
• assign_to_agent family — Issue Monster present but read-only; not exercised in a failing path.

Recommendations

Critical / High

1. Unify missing-context handling across safe-output handlers (Priority: High) — Make update_pull_request and add_comment soft-skip (like create_pull_request_review_comment) when no triggering context and no explicit resolvable target exists, instead of hard-failing the job. Root cause of all 3 recent PR Sous Chef failures.
2. Pre-bundle the Process Safe Outputs step log on failure (Priority: High) — Every in-scope failure for weeks has had unrecoverable exact error text. Without this, root cause stays inferred.

Medium

3. PR Sous Chef payload hygiene — Have the workflow attach an explicit pull_request_number per update_pull_request/add_comment item, or gate those writes on a resolvable target, so scheduled runs stop redding.

Work Item Plans

WI-1 — Soft-skip target-requiring writes on missing context (Bug Fix, High)

• Acceptance: a scheduled run emitting add_comment/update_pull_request with no resolvable target soft-skips those items (⏭) and the safe_outputs job concludes success; regression test covers schedule-trigger + mixed payload.
• Approach: align update_pull_request/add_comment handlers with the create_pull_request_review_comment soft-skip branch. Effort: Small–Medium.

WI-2 — Pre-bundle Process Safe Outputs step log on failure (Enhancement, High)

• Acceptance: on safe_outputs job failure, the step stdout/stderr is uploaded as an artifact and visible to the audit tooling.
• Effort: Small.

Historical Context / Trends

• Trend: Clean streak (07-21/07-22) broken again — same "write-rich window breaks the streak" pattern as 07-20 and 07-23.
• Shift in dominant signature: The scheduled-target-resolution family (PR Sous Chef) has overtaken the review-comment family as the most reliable in-scope production failure signature — 3 hard-fails in 2 days, all same workflow + shape.
• Rolling in-scope success: 07-20 94.6% → 07-21 100% → 07-22 100% → 07-23 99.24% → 07-24 97.44%.

Metrics & KPIs

• Overall in-scope safe-output success rate: 97.44%
• Most problematic: update_pull_request / add_comment on scheduled runs (PR Sous Chef)
• Most reliable: read-only reviewer jobs (all succeeded)

Next Steps

• Open/attach tracking issue for WI-1 (handler soft-skip unification) — now the top recurring production signature
• Open/attach tracking issue for WI-2 (pre-bundle step log)
• Watch next PR Sous Chef scheduled write-actuating run for recurrence
• Continue tracking Changeset Generator patch-format:BUNDLE (unvalidated ~28d) and review_path_unresolved_422 Path-variant (unvalidated 56 audits)

References:

• §30057263862 — PR Sous Chef failure rejig docs #1
• §30068020977 — PR Sous Chef failure Add workflow: githubnext/agentics/weekly-research #2
• §29974524390 — PR Sous Chef 07-23 (same shape)

Generated by 🔒 Safe Output Health Monitor · age00 · 496.3 AIC · ⌖ 16.7 AIC · ⊞ 9.4K · ◷

• expires on Jul 24, 2026, 9:29 PM UTC-08:00

[github-actions[bot]]

This discussion was automatically closed because it expired on 2026-07-25T05:29:58.976Z.

Closed by Workflow