Emit a standalone parser, equal to the interpreter — + integer token kinds + sticky/dispatch lexer (~5.0x the official TS parser)

[johnsoncodehk]

What

createParser(grammar) interprets the RuleExpr trees at runtime — it's the one derived output that isn't emitted as code (the TextMate / Monarch / tree-sitter highlighters all are). This PR adds emitParser(grammar) (src/emit-parser.ts): a self-contained JS parser that bakes the grammar's static analysis into constants and specializes each rule into straight-line code, removing the central matchExpr tree-walk. It reuses the lexer runtime — so the parser is what's specialized — and never imports the grammar-definition object. Then two perf levers, both proven byte-identical against createParser as the oracle:

1. Stage A — the emitter. Per-rule + per-arm specialized matchers; the Pratt / left-rec / longest-match / mixfix / memo cores stay as a shared runtime the emitted code calls.
2. Integer token kinds (emit-parser.ts). String-keyed dispatch (tok.type === key, Set.has, startsWith) → integer compares, via a two-int interning pass after tokenize (type-kind + literal-kind; two fields because a keyword is an identifier-with-text here, so it must match both its keyword key and the identifier token-name key).
3. Sticky-regex + first-char dispatch lexer (src/gen-lexer.ts, shared — also speeds up the interpreter and every language). tokenize no longer tries every matcher per position against a fresh source.slice(pos): matchers are sticky regexes exec'd in place, and a conservative first-char analysis skips matchers whose possible first chars can't match the current char (so (/; skip the word/number/string regexes).

Measured (TS grammar)

• Correctness: byte-identical CST + accept/reject vs createParser across the full tests/cases corpus — 18,805 / 18,805 at every step. The shared lexer change is byte-identical too: identical token stream across TS+JS+YAML corpora (0 differing tokens of ~54k) and conformance unchanged (run-conformance 5386/5659).

• Emitted parser, four bench files (best-of-5):

  	emitted	vs official tsc
  Stage A	50.9 ms	6.3×
  + integer kinds	42.6 ms	5.6×
  + sticky/dispatch lexer	~38 ms	~5.0×

  vs the interpreter the emitted parser is now ~1.9×; the lexer lever (~1.25× on tokenize) also speeds up the interpreter, so both improve. tsc ≈ 7.6 ms.

• Context: the gap to the official parser is implementation, not a V8 ceiling — tsc is itself JS. The decomposition is even (lexer ≈ parser layer); the remaining lexer cost is regex-per-token execution vs tsc's hand-tuned char scanner.

Scope

src/emit-parser.ts (new) + src/gen-lexer.ts (shared lexer; byte-identical token stream, verified across languages) + test/emit-parser-verify.ts (correctness gate) + test/emit-parser-bench.ts (benchmark). createParser (the oracle), the grammars, and the generated artifacts are untouched.

Follow-ups (tracked as issues)

• Emit the lexer: author tokens as a target-neutral IR instead of regex strings #5 — Emit the lexer: author tokens as a target-neutral IR (combinators) instead of regex strings, deriving a char-code DFA scanner. The next ceiling (the lexer is still the reused regex runtime) and what makes the parser self-contained for Go/Rust.
• Target-agnostic emitter: emitParser(grammar, target) for Go/Rust/native #6 — Target-agnostic emitter: emitParser(grammar, target) with JS-specific emission behind a Target config (Go / Rust / native).
• CI: gate the emitted parser against createParser (run emit-parser-verify) #7 — CI gate: run emit-parser-verify so emitted == createParser can't silently rot.
• Reduce parser-layer backtracking / longest-match (remaining gap to the official parser) #8 — Reduce parser-layer backtracking / longest-match (the remaining parser-side gap to the official parser).

Notes

The emitted parser (~290 KB for TS) is written to /tmp by the scripts and is not committed. Subtleties handled: parseRule sets the Pratt context + packrat memo for both Pratt and left-rec rules (left-rec rules go through the same wrapper, else ${…} in a template-literal type resolves to Expr not Type); the int-kinds vocabulary must be a superset of every dispatch key, including operators that appear only inside a not (reserved-word) lookahead; the first-char dispatch is conservative (proven first-char superset, verified by a 5k-input fuzz + full token byte-identity).