Skip to content

check.sh: a partial run is indistinguishable from a full one on stdout, and no test guards it #336

Description

@JArmandoAnaya

Problem

Rewritten 2026-08-05 after the backlog audit: the original framing — "stops silently",
and a choice between installing and failing loudly — described behaviour that had already
been fixed. PR #249 (a4c7b44, 2026-08-02) shipped the fail-loudly guard three days
before this issue was filed
, and two days before the #329/#333 run cited as the
observation. What survives is narrower, and real.

scripts/check.sh aborts correctly when node_modules is missing — but it says so on
stderr and puts nothing at all on stdout
, so a caller reading stdout sees the pre-fix
symptom exactly: green pytest output, then nothing. No verdict line, no timing table, no
marker of any kind distinguishes a run that covered one group from a run that covered four.

scripts/check.sh:114-119, called at the head of run_frontend (:122), run_generated
(:139) and run_browser (:171):

require_node_modules() {
  if [[ ! -d node_modules ]]; then
    echo "error: node_modules is missing — run 'pnpm install' first" >&2
    exit 2
  fi
}

Measured against a copy of the script in a directory with no node_modules:

$ bash <copy>/scripts/check.sh generated 2>/dev/null ; echo exit=$?
exit=2
$ bash <copy>/scripts/check.sh generated ; echo exit=$?
error: node_modules is missing — run 'pnpm install' first
exit=2

The exit code is right and All checks passed. is structurally unreachable — it is printed
only after the group loop completes (check.sh:232). The gap is that stdout carries no
evidence either way
, which is how an agent or a CI step that captures stdout still reads a
partial run as a calm one. This is the same class of false-calm failure the script's own
header warns about for | tail.

What is left to do

  1. Put the abort on stdout as well, or emit a stage-coverage line the reader cannot
    miss — a partial run must be distinguishable from a full one in the output a caller
    actually captures
    , not only in the exit code and stderr.
  2. Guard it with a test, so a future edit cannot quietly reintroduce a partial pass that
    looks complete.

Acceptance

  • A test or assertion guards the stage count / completion marker, so a future
    edit cannot reintroduce a quiet partial pass

(The original first criterion — "either completes all stages or exits non-zero naming the
prerequisite" — is already met by PR #249 and has been removed.)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions