[safe-output-health] π₯ Safe Output Health Report β 2026-06-11 (clean streak broken: 3 safe-output job failures) #38514
Closed
Replies: 2 comments
-
|
Cave bot tap drum. Smoke run on PR #38506. Fire still burn. Warning Firewall blocked 5 domainsThe following domains were blocked by the firewall during workflow execution:
network:
allowed:
- defaults
- "accounts.google.com"
- "clients2.google.com"
- "contentautofill.googleapis.com"
- "safebrowsingohttpgateway.googleapis.com"
- "www.google.com"See Network Configuration for more information.
|
Beta Was this translation helpful? Give feedback.
0 replies
-
|
This discussion was automatically closed because it expired on 2026-06-12T06:11:29.070Z.
|
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
π₯ Safe Output Health Report β 2026-06-11
All three failures share one root cause family: safe-output target/context resolution in non-issue/non-PR trigger contexts (
schedule/workflow_dispatch), handled inconsistently across handlers β some soft-skip, others hard-fail the job.Executive Summary
update_issue, 1Γadd_labels, 1Γremove_labels, 1Γadd_comment)Methodology: every
conclusion=failurerun was audited via theauditMCP tool and classified by its job breakdown (agent/detection/safe_outputsconclusions). For the 3 in-scope failures, the downstream "Process Safe Outputs" step logs were read directly to extract exact per-message errors.Critical Finding (Production) β LintMonster
update_issuehard-fails on scheduled runRun Β§27322319441 Β· trigger
scheduleΒ· jobs:agent=success,detection=success,safe_outputs=FAILURE (25s) Β· Result: Successful: 2, Failed: 3.create_issueβ [lint-monster] chore: Replace sort.Slice with slices.SortFunc for type-safety (3 issues)Β #38494,create_discussionβ [lint-monster] Daily Lint Scan Report β Jun 11, 2026Β #38495.update_issueops targeting tracking issues [lint-monster] chore: Function length refactoring (645 functions exceed 60-line limit)Β #38269 / [lint-monster] chore: Replace map[string]bool with map[string]struct{} (152 instances)Β #38270 / [lint-monster] chore: Fix context propagation and os.Setenv (11 issues)Β #38271.update-issue:config (.github/workflows/lint-monster.md:64-66) sets onlymax: 10+title-prefix, with notarget: "*"β so it defaults totarget: "triggering". LintMonster is schedule-triggered, so there is no triggering issue. Critically, the agent did supply explicitissue_number(38269/38270/38271) on each message, but the handler resolved against the default target and ignored the explicit number.##[warning]...skipping...(soft intent) yet records##[error]...failed(hard accounting), failing the whole job.Smoke-Test Findings (Lower Severity, By-Design Edge Cases)
Smoke Copilot run-27320477001 β add_labels / remove_labels "No issue/PR number available" (Failed: 2)
Run Β§27320477001 Β·
workflow_dispatchΒ·agent=success,detection=success,safe_outputs=FAILURE;send_slack_message+update_cache_memorysucceeded.This run is the clearest proof of the handler inconsistency β under the identical missing-context condition, the review-comment handler soft-skipped:
...while
add_labels/remove_labelshard-failed. (Later messages 16/17 with explicit numbers succeeded; the agent even emittedreport_incompletenoting labels readback was unchanged.)Smoke Copilot run-27315898580 β add_comment "target must be one of: [status]" (Failed: 1)
Run Β§27315898580 Β·
workflow_dispatchΒ·agent=success,detection=success,safe_outputs=FAILURE.The
add_commentmessage carried atargetvalue outside the workflow's configured allowed set[status]β a validation-time target rejection. Same run also soft-skippedcreate_pull_request_review_commentandreply_to_pull_request_review_comment(β) for missing PR context.Root Cause Analysis β One Unifying Theme
All three failures are the target/context-resolution family first flagged on 2026-05-22 (
target_star_review_comment_no_pr_number_fallback) and 2026-05-27 (target_star_add_comment_no_item_number_fallback). The recurring defect is that handlers disagree on what to do when the trigger context is missing/unresolvable:create_pull_request_review_commentreply_to_pull_request_review_commentupdate_issueissue_numberadd_labels/remove_labelsadd_commentRecommendations
Critical (production):
target: "*"to theupdate-issue:block in.github/workflows/lint-monster.mdand recompilelint-monster.lock.yml. The agent already supplies explicitissue_number, sotarget: "*"lets the scheduled run update arbitrary issues by number.update_issueSystem-side (handler consistency β addresses all three + the recurring family):
issue_numberover the default target. In theupdate_issuehandler, when a message carries an explicitissue_number, use it regardless oftarget: "triggering"(an explicit number is an explicit target). Only fall back to triggering-context resolution when no number is given.update_issue,add_labels, andremove_labelssoft-skip (β) when no target is resolvable β matchingcreate_pull_request_review_commentβ instead of##[error]that fails the job. Reconcile the contradictorywarning: "skipping"+error: "failed"wording.update_issue,add_labels,remove_labelsSmoke tests (low priority): the two Smoke Copilot failures are by-design edge-case exercises; once the handlers soft-skip, they will stop reddening the smoke suite. No separate action needed beyond rec. #3.
Work Item Plan
WI-1: LintMonster
update-issuetarget config (Bug Fix Β· High)target: "*"added + recompiled; next scheduled LintMonster run updates [lint-monster] chore: Function length refactoring (645 functions exceed 60-line limit)Β #38269/[lint-monster] chore: Replace map[string]bool with map[string]struct{} (152 instances)Β #38270/[lint-monster] chore: Fix context propagation and os.Setenv (11 issues)Β #38271 withsafe_outputs=success.lint-monster.mdfrontmatter, run the compiler.WI-2: Handler missing-context consistency (Enhancement Β· Medium)
update_issue/add_labels/remove_labelssoft-skip (β, job stays success) on missing trigger context; explicitissue_numberhonored onupdate_issue; warning/error wording reconciled. Unit tests cover the missing-context soft-skip path.safe_output_handler_manager.cjstarget-resolution branches with the review-comment handlers.Historical Context & Trend
review_path_unresolved_422Path-variant fallback (pr_review_buffer.cjs:554) remains UNVALIDATED for the 14th consecutive audit (no Path/Line 422 surfaced; both Smoke Copilot runs soft-skipped review comments).target_star_*) has now produced hard failures on 2026-05-22, 2026-05-27, and 2026-06-11, expanding from review-comment β add_comment β update_issue/add_labels/remove_labels. The fix is structural: one shared missing-context policy across handlers.Metrics
update_issue(3 failed messages, production impact).safe_outputs=success (clean failure-path handoff).Next Steps
target: "*") and verify next scheduled run.review_path_unresolved_422Path-variant to finally exercise its fallback.References:
update_issuefailure)Beta Was this translation helpful? Give feedback.
All reactions