Skip to content

Fix flaky SIGPIPE/pipefail false positives in safe-outputs conformance checker - #58768

Merged
pelikhan merged 2 commits into
mainfrom
copilot/req-001-fix-conformance-checker
Sep 5, 2026
Merged

Fix flaky SIGPIPE/pipefail false positives in safe-outputs conformance checker#58768
pelikhan merged 2 commits into
mainfrom
copilot/req-001-fix-conformance-checker

Conversation

Copilot AI commented Sep 5, 2026

Copy link
Copy Markdown
Contributor

scripts/check-safe-outputs-conformance.sh produced non-deterministic MEDIUM failures (e.g. REQ-001) against an unmodified spec file due to a SIGPIPE race under set -o pipefail.

Root cause

Several checks piped a bounded grep -A N into a grep -q/grep -qE. When the downstream grep matches and exits early, it can send SIGPIPE to the still-writing upstream grep, which then exits 141 — surfacing as pipeline failure under pipefail even though the keyword was found.

Fix

  • Replaced each grep -A N ... | grep -q ... pipeline with a two-step pattern: capture the first grep's output via command substitution ($(... || true)), then match against it with grep <<< "$var". This runs the first grep to completion before matching, eliminating the race.
  • Applied consistently across all affected call sites: SEC-001 (privilege separation), REQ-001 (RFC 2119 keywords), REQ-002 (type completeness), REQ-003 (verification methods), MCE-001 (tool-description constraints), plus two additional occurrences of the same anti-pattern found during audit (TYPE-008, TYPE-010) not explicitly listed in the issue.
# before (racy)
if ! grep -A 200 "## .*$section" "$spec_file" | grep -q "MUST\|SHALL\|SHOULD\|MAY"; then

# after (deterministic)
section_text=$(grep -A 200 "## .*$section" "$spec_file" 2>/dev/null || true)
if ! grep -q "MUST\|SHALL\|SHOULD\|MAY" <<< "$section_text"; then

Verified by running the script 10 consecutive times against the unmodified repo — identical PASS results every run.

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix non-deterministic false positives in conformance checker Fix flaky SIGPIPE/pipefail false positives in safe-outputs conformance checker Sep 5, 2026
Copilot AI requested a review from pelikhan September 5, 2026 08:47
@pelikhan
pelikhan marked this pull request as ready for review September 5, 2026 08:48
Copilot AI balanced review requested due to automatic review settings September 5, 2026 08:48
@pelikhan
pelikhan merged commit b5cd489 into main Sep 5, 2026
1 check passed
@pelikhan
pelikhan deleted the copilot/req-001-fix-conformance-checker branch September 5, 2026 08:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

The changes consistently remove the identified race without altering the checks’ intended semantics.

Pull request overview

Eliminates SIGPIPE/pipefail races in the safe-outputs conformance checker.

Changes:

  • Captures bounded grep output before matching.
  • Applies the deterministic pattern across seven affected checks.
File summaries
File Description
scripts/check-safe-outputs-conformance.sh Removes racy grep pipelines from conformance checks.
Review details
  • Files reviewed: 1/1 changed files
  • Comments generated: 0
  • Review effort level: Balanced

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Safe Outputs Conformance] REQ-001: Conformance checker produces non-deterministic false positives (pipefail/SIGPIPE race)

3 participants