Skip to content

The shared textual scanners open a string on a quote inside a REGEX literal - 46 code: positions in 68 files stay unplaceable after #14626 #14742

Description

@claude

Found while closing #14626 (the nested-template desync in the same four primitives). Out of scope there: it is a different literal class with a different fix, and #14626's scope ruling named the template-literal mode only. Measured on the same sweep, and recorded in SCANNER_LITERAL_BLIND_SPOTS in scripts/check-dispatcher-error-vocabulary.mjs rather than left to be rediscovered.

The defect

skipStringLiteral (and therefore scanTopLevel, enclosingOpeners, sliceBalanced and splitTopLevel) opens a string on any ', " or backtick. A regex literal whose character class contains one of those three opens a literal the source does not have, and the scan is inside out from there. The live instance is in packages/rest/src/error-response.ts:

const RELATION_DOES_NOT_EXIST = /\brelation\s+["'`][^"'`]+["'`]\s+does not exist/i;

The walk enters at the " inside the character class. Every position after it in that file is read with the string/code polarity reversed.

What it costs, measured

Over packages/** non-test source (2187 files, 906 code: tokens), after #14626's template-literal mode landed:

files whose whole-file walk is health-certified (no unterminated literal, no bracket underflow, empty stack at EOF) 2119
files still desynchronised 68
code: positions the scan cannot place and cannot certify 46
of the unplaced, positions certified as genuinely inside a string (the correct answer) 21
positions at objlitexpr's anchor sitting behind this class 11
unregistered wire codes hiding behind it today not knowable without closing it

Nothing is red today: #14626's sweep diffed deriveSites key-by-key and no site or unresolved entry moved. The cost is the same one #14626 was graded p2 for — a count the gate prints as a total is a floor — except that this half is now named and counted instead of silent, which is the improvement #14626 shipped rather than the defect it closed.

Why it was not fixed in #14626

⛔ It is not mechanical. Telling /re/ from division needs the preceding token's grammatical class (a / b / c and x = /b/ differ only in what came before the slash), which is lexer state this deliberately textual scan does not carry — the file's own "Why textual, not AST" note is the reason it does not. A heuristic bolted on here would trade a blind spot that is now COUNTED for one that is not, which is the wrong direction, so #14626 counted it and stopped.

What a fix would owe

  • A minimal regex-literal recogniser in skipStringLiteral, with the preceding-significant-token rule stated from the grammar rather than approximated, plus the division cases as negative fixtures.
  • A --self-test case with a positive control, exactly as A nested template literal desynchronises the vocabulary gate's shared textual scanners, and every code: position after it in that file goes unplaced #14626's four primitives got: the pin already exists in the opposite direction (a regex fixture must currently come back desync), so closing this class reds that pin — which is the point, and the pin's message says to re-measure rather than edit it.
  • A re-derivation of SCANNER_LITERAL_BLIND_SPOTS, INLINE_LITERAL_EXPRESSION_CENSUS and OBJECT_LITERAL_CODE_HELPER_BLINDNESS.localTwinCensus, since all three publish figures that move when 46 positions become placeable.
  • Any newly surfaced unregistered value is a separate registration decision, not a rider.

Builds on #14626 (its skipStringLiteral, its scan-health certificate and its blind-spot block are what make this measurable at all). Unassigned, for triage.


Generated by Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions