Skip to content

[finding] measure-self-test-floor's --probe has zero coverage of the helper handshake spelling — its only carrier is an ENTRY_BY_HAND null #15371

Description

@claude

Measured while landing #15324 (the probe's HELD-needs-a-printed-handshake fix) on c4d1354e3. Filed unassigned, as a finding only — no severity asserted, grading is triage's.

The reading

--probe in scripts/measure-self-test-floor.mjs has zero dynamic coverage of one of the three landed handshake spellings. Measured on c4d1354e3, whole population, 170 rows:

handshake spelling carriers under scripts/ probe verdict coverage
sentinel — SELF_TEST_VERDICT 87 exercised (many HELD rows)
flag — selfTestReachedVerdict 78 exercised (many HELD rows)
helper — requireReachedVerdict 1 none

The helper form's only carrier in the tree is scripts/check-platform-checklist.mjs, and that file carries an ENTRY_BY_HAND row of null, so every probe run records:

{"verdict":"NOT MEASURED","why":"entry read by hand as not probeable -- see ENTRY_BY_HAND"}

The null is correct and deliberate — its dispatch calls four self-test functions and combines their statuses, so a one-function early return measures a sub-battery and would read a downstream crash as a handshake. Nothing here asks for that row to change.

Why it is worth recording

The consequence is about what a green sweep is evidence FOR. --probe is quoted as the completion picture for #13798/#13799, and for the helper spelling it says nothing at all — not "held", not "defeated", but no reading in either direction. So:

The instrument's inline controls do exercise a sentinel-shaped fixture (SOUND_GATE) in both directions, so the sentinel shape has a permanent control. The helper shape has neither a control fixture nor a live row.

Options

  • A. Add a helper-shaped fixture to the instrument's inline controls. Cheapest, and it follows the file's own convention (controls run on every invocation, never behind a flag). It proves the probe can read that shape, without needing a probeable carrier in the tree. Cost: a third fixture string.
  • B. Make check-platform-checklist.mjs probeable — e.g. teach ENTRY_BY_HAND to name a list of entries and combine their probes. Real coverage of the real file; considerably more instrument. Overlaps The self-test-floor census cannot measure check-workspace-manifest-cycles.mjs - two self-test-shaped functions and no ENTRY_BY_HAND row, so its handshake reads NOT MEASURED #14842's shape question.
  • C. Record the gap in the instrument's header and leave the coverage at zero. Zero cost, and the NOT MEASURED row is already honest; but "which spellings has this sweep actually exercised" stays a question you answer by reading.

Recommendation: A. The claim a control can support here is "the probe reads this shape", and a fixture supports it without waiting on B; C leaves the reader to re-derive this table by hand, which is the recurring cost #14968 is about.

Related

#14968 (no canonical way to ask whether a self-test has a handshake; the three-spelling table), #14842 (two self-test-shaped functions and the ENTRY_BY_HAND row), #14963 (the probe anchors on the first selfTest match), #15324 (this probe's verdict).

Duplicate-checked before filing against 380 open issues read from the repo-scoped issues list plus a local grep; control terms returned the known family (#14842 / #14963 / #14968 / #15296 / #15339), confirming a live read. No existing card asks about per-spelling probe coverage.


Generated by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions