Skip to content

test(compat): the fail-on-red gate's node -e invocation must be top-level (#704) - #706

Open
AhmedElBanna80 wants to merge 2 commits into
mainfrom
fix/704-gate-invocation-top-level
Open

test(compat): the fail-on-red gate's node -e invocation must be top-level (#704)#706
AhmedElBanna80 wants to merge 2 commits into
mainfrom
fix/704-gate-invocation-top-level

Conversation

@AhmedElBanna80

Copy link
Copy Markdown
Collaborator

Closes #704. Also closes #702 — see below; that one needed no code.

The defect

#701 made the gate's three teeth independently provable inside the step's script, and gave the JS half a "must be top-level" assertion. The shell half had no sibling. Wrapping only the node -e invocation — leaving the prelude untouched — disarms all three teeth:

if [ "${KNEXT_RUNTIME:-node}" != "bun" ]; then
  node -e '…' "${SUMMARY}"
fi
audit real step
baseline teeth 0/0/0, branches 3, reaches 0, escape 0 exit=1, error lines present
wrapped byte-identical exit=0, 0 ::error::red shard lines

Measured with bash -e, KNEXT_RUNTIME=bun, a red 8-failure Bun-lane summary. A night with real test failures concludes SUCCESS and counts toward the 14-night window in docs/compat/window-node-lane.md.

It slips every existing guard by construction. Wrapping the whole prelude is already caught — shellIfBlocks skips nested blocks via i = j, so the missing-summary branch stops being found. The narrower edit survives that. And it introduces no exit, so the escape-hatch scan has nothing to find.

The fix

shellNestingDepthAt nets block openers against closers per line, so anything opening and closing on one line contributes zero — which keeps "${SUMMARY}" and [ ! -f … ] from registering as nesting. The embedded program is blanked by offset first, or its JS braces would count as shell blocks.

Five shapes now red: conditional, brace group, subshell, case arm, function body. Baseline 0.

The near-miss, recorded because it is the point

The first cut caught four of five and missed the case arm. case … in opened the block, and the very next pattern line — node) — closed it again, netting zero at the invocation. The other four were caught.

The shape that would have shipped broken is the one nobody probes. That is the same failure eleven rounds of review on #701 kept surfacing, and it was caught here only by running all five shapes rather than the one that motivated the fix. Hence CASE_PATTERN, and hence the five-way probe.

#702 needs no code — its subject no longer exists

#702 reported that blankSingleQuoted's even-quote parity check is necessary but not sufficient, so a || true bracketed by two stray apostrophes is blanked and survives.

blankSingleQuoted was deleted by #701 round 9. No quote-pairing logic remains in tests/helpers/fail-on-red-gate.ts — blanking is offset-based (embeddedNodeProgramSpan) and exit is matched only in command position; neither depends on pairing.

Verified empirically rather than by reading, using #702's own reproduction:

BASELINE                            reaches 0
CONTROL  || true alone              reaches 1
STRAY    two apostrophes + || true  reaches 1   ← was the escape; now reds

Evidence

  • Mutation-proved: the wrap reds expect(escapeHatches).toEqual([]); restore byte-exact; scan-mutation-residue.mjs clean; baseline green.
  • Rides the existing assertion rather than adding a competing one, so no new red can stand in for another.
  • tests/compat-suite-workflow.test.ts + tests/compat-shard-flake-attribution.test.ts: 153 passed.

What I did not verify

…evel (#704)

The JS half already asserts each tooth's owning `if` is a top-level statement of
the embedded program. The shell half had no sibling, and the asymmetry was live.

Wrapping ONLY the invocation — leaving the prelude alone — disarms all three
teeth while every existing signal stays byte-identical to baseline:

    if [ "${KNEXT_RUNTIME:-node}" != "bun" ]; then
      node -e '…' "${SUMMARY}"
    fi

    audit: teeth 0/0/0 · branchesFound 3 · exitReachesStep 0 · escapeHatches 0

Measured against the real step (`bash -e`, KNEXT_RUNTIME=bun, a red 8-failure
Bun-lane summary): exit 1 → exit 0, zero `::error::red shard` lines. A night with
real test failures concludes SUCCESS and counts toward the 14-night window.

It slips every existing guard by construction. Wrapping the WHOLE prelude is
already caught, because `shellIfBlocks` skips nested blocks (`i = j`) so the
missing-summary branch stops being found — it is the narrower edit that survives.
And it introduces no `exit`, so the escape scan has nothing to find.

`shellNestingDepthAt` nets block openers against closers on each line, so a
construct that opens and closes on one line contributes zero — which is what
keeps `"${SUMMARY}"` and `[ ! -f … ]` from registering as nesting. The embedded
program is blanked by offset first, or its JS braces would count as shell blocks.

MEASURED, NOT ANTICIPATED — the first cut caught four of five shapes and missed
the `case` arm: `case … in` opened the block and the very next pattern line,
`node)`, closed it again, netting zero at the invocation. Conditional, brace
group, subshell and function body were all caught. The shape that shipped broken
would have been the one nobody probed, which is why `CASE_PATTERN` exists and why
all five are exercised rather than the one that motivated the fix.

Rides the existing `expect(escapeHatches).toEqual([])` assertion rather than
adding a competing one. Mutation-proved: the wrap reds that assertion, restore
verified byte-exact, residue scan clean, baseline green.

Closes #704

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198oVxWkEEDJJnVhES6ULnx
@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
knext-docs Ready Ready Preview Aug 6, 2026 4:28pm

…ction (#704)

Self-review found a limit I had not documented: an unbalanced `(` or `{` inside
a QUOTED string counts as an opener that never closes. Measured — `echo "a
smiley ( in a string"` nets +1 — so two such lines before the invocation would
report nesting that is not there.

The DIRECTION matters and is stated explicitly, because this file got exactly
that backwards once before (#701 round 9, where a fail-open/fail-closed note was
inverted and the "never" was false). This one is **fail-CLOSED**: the only
reachable error is a spurious finding, never a missed disarm.

That is the opposite direction from #702, whose concern was fail-OPEN — a real
`|| true` hidden by mispaired quotes. Conflating the two is how a limitation gets
waved through as already-known, so the note says which is which.

Not live: the gate's prelude carries no unbalanced bracket inside a string, and
the baseline measures 0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0198oVxWkEEDJJnVhES6ULnx
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant