Skip to content

Release 4.9.0

Choose a tag to compare

@kaby76 kaby76 released this 08 Nov 03:17
· 20 commits to main since this release

Release 4.9.0

This branch adds persistent DFA warm-up for interpreted parsing, reduces ALL(*) parser and tree-building overhead, and fixes Kotlin grammar interpretation. It also updates the tool versions and adds a proposal for describing language features outside grammar files.

Interpreter DFA warm-up

  • trparse --write-dfa-cache captures learned lexer and parser DFAs in the output PAX/tar bundle alongside the .interp tables. A subsequent trparse --allstar command reads the tables and cache from stdin before parsing its input files. --write-dfa-cache-file FILE can also save a standalone copy.
  • --write-dfa-cache --no-output creates a compact warm-up bundle without the parsed input's .pt and .errors results. Bundle reading stages only interpreter tables and cache data rather than materializing unrelated parse-result payloads.
  • Cache snapshots have a format version and SHA-256 checks for both interpreter tables. Parser DFA reuse respects the existing sharing limits and can be disabled independently while retaining the lexer DFA. Cache loading and writing are included in interpreter timing diagnostics.
  • Documentation and regression tests cover cache round-trips, mismatched tables, bundle member ordering, and warm-up bundles.

ALL(*) parser and tree construction

  • Replace the active-rule-call hash-set hot path with per-rule stack frames that detect the same recursive call at the same token position and precedence. Keep a hash-guard comparison path in tests; add DOT and indirect-left-recursion coverage.
  • Build the compact parse tree directly from committed parse events, avoiding retention of a complete intermediate event list for ordinary ALL(*) parsing. Add tree-equivalence and DOT performance tests.

Kotlin grammar interpretation

  • Resolve lexer grammar imports in trinterp, including imported fragments, modes, channels, and token-type mappings. Add the Kotlin grammar and fix-3218.kt regression case.
  • Replace the lexer's 32-bit action mask with ordered, interned action sequences. Kotlin's Inside_Identifier -> type(Identifier) command was above index 31 and had been silently omitted, causing fix-3220.kt to fail. The new representation also updates the persisted DFA cache format. Add token-type and parse regression coverage for that input.

Generated target fixes

  • Correct Dart driver output paths on Windows: input filenames remain runtime arguments, while an optional --input-root preserves example subdirectories under -o. Update the generated Bash and PowerShell test scripts to pass the root at runtime. Freshly generated Kotlin Dart drivers compiled, and make test passed on Windows.
  • Update the PHP generated app's phpunit/php-timer dependency from ^7.0 to ^9.0.

Other changes

  • Set the tool and package versions to 4.9.0 and remove temporary debugging output.
  • Add proposals/assignment-semantics.md, describing an optional mapping from grammar constructs to language-feature categories, starting with assignment syntax.

Verification

  • dotnet build src/trparse/trparse.csproj -c Release --no-restore passed.
  • The focused Kotlin, lexer-command, and interpreter bundle/DFA-cache suite passed: 26 tests.
  • The rebuilt ALL(*) interpreter parsed the formerly failing Kotlin fix-3220.kt fixture.

See PR #757 for the branch changes.