Prune longest-match alternatives by their deep FIRST set (#8)

[johnsoncodehk]

What

The longest-match loops — parseNonRec, the parseLeftRec atoms, the parsePratt nuds — try every alternative the lookahead doesn't rule out and keep the longest. The "rule out" test was shallow (firstTokenOf): it pruned an alternative only when its FIRST element was a literal or a token ref. A leading rule ref (Decl …, Expr …) defeated it → null → always tried. So a rule-ref-led alternative was speculatively parsed (and usually failed) at every position.

Instrumented on the TS corpus, that's the bulk of the waste: the NUD/atom loops try ~4.4 alternatives per dispatch, ~80% of those matchExpr attempts never win the longest-match, and ~57% of all attempts are alternatives a deep FIRST set would have ruled out (70% of the wasted ones).

This PR resolves each alternative's FIRST set through the transitive firstSets (already computed for the rule-ref guards) and skips the alternative when the lookahead is provably outside it — issue #8's lever 2 ("strengthen first-token dispatch so provably-dead alternatives are never entered").

Sound by construction (zero behaviour change)

exprFirst is a complete over-approximation — it never omits a token a non-empty match could begin with — so an alternative pruned here genuinely cannot match non-empty at this token. A nullable alternative is always tried: its only extra match is the empty one, and an empty match never wins the longest-match comparison (pos === saved, never > bestPos). The deep test strictly dominates the shallow one (whenever the shallow check pruned, the deep FIRST set — whose leading member is that same literal/token — prunes too).

Verified:

check	result
interpreter CST hash (809-file sample) vs HEAD	identical (full CST JSON + accept/reject)
createParser ≡ emitParser, full corpus	18,805 / 18,805 byte-identical, 0 CST mismatch
run-conformance accept/reject set	unchanged, 5386/5659
npm run check	26/26 gates

Mirrored in both engines: createParser (gen-parser.ts, altMightStart) and the emitted emitParser (emit-parser.ts, altGuard). The per-alt FIRST descriptor arrays are precomputed (interpreter) / hoisted to deduped module-level consts (emitter), so the guard allocates nothing per call — which also removes the same per-call array allocation from the existing rule-ref firstGuard, and shrinks the emitted TS parser 328 KB → 314 KB.

Measured

Isolating the parser layer (full parse − tokenize, since the lexer is untouched), interleaved best-of-N vs the prior engine on the four bench files:

	parser layer old → new
parserharness.ts	10.9 → 10.0 ms (~9%)
fixSignatureCaching.ts	2.1 → 1.8 ms (~15%, now 0.92× tsc — faster than tsc on this file)
parserRealSource7.ts	3.7 → 3.5 ms (~6%)
parserindenter.ts	3.0 → 2.8 ms (~6%)
aggregate	19.8 → 18.2 ms (~9%)

Full-pipeline that's ~4–5%, since (per #4) the lexer is ~half the time. The headroom here is inherently small: the parser layer is already ~1.6× tsc and on some files beats it — the dominant remaining gap is the lexer (regex-per-token vs tsc's hand-tuned char scanner), which is #5's territory.

Scope

src/gen-parser.ts + src/emit-parser.ts only. The grammars, generated artifacts, and createParser's public behaviour are untouched. Lever 1 (committing through genuine shared-first-token ambiguity — (→paren/arrow, ident→ident/arrow) is not attempted: FIRST sets can't separate those, and with the parser layer already near tsc the headroom doesn't justify the structural risk.