Repository navigation
Release 4.9.0
Release 4.9.0
This branch adds persistent DFA warm-up for interpreted parsing, reduces ALL(*) parser and tree-building overhead, and fixes Kotlin grammar interpretation. It also updates the tool versions and adds a proposal for describing language features outside grammar files.
Interpreter DFA warm-up
trparse --write-dfa-cachecaptures learned lexer and parser DFAs in the output PAX/tar bundle alongside the.interptables. A subsequenttrparse --allstarcommand reads the tables and cache from stdin before parsing its input files.--write-dfa-cache-file FILEcan also save a standalone copy.--write-dfa-cache --no-outputcreates a compact warm-up bundle without the parsed input's.ptand.errorsresults. Bundle reading stages only interpreter tables and cache data rather than materializing unrelated parse-result payloads.- Cache snapshots have a format version and SHA-256 checks for both interpreter tables. Parser DFA reuse respects the existing sharing limits and can be disabled independently while retaining the lexer DFA. Cache loading and writing are included in interpreter timing diagnostics.
- Documentation and regression tests cover cache round-trips, mismatched tables, bundle member ordering, and warm-up bundles.
ALL(*) parser and tree construction
- Replace the active-rule-call hash-set hot path with per-rule stack frames that detect the same recursive call at the same token position and precedence. Keep a hash-guard comparison path in tests; add DOT and indirect-left-recursion coverage.
- Build the compact parse tree directly from committed parse events, avoiding retention of a complete intermediate event list for ordinary ALL(*) parsing. Add tree-equivalence and DOT performance tests.
Kotlin grammar interpretation
- Resolve lexer grammar imports in
trinterp, including imported fragments, modes, channels, and token-type mappings. Add the Kotlin grammar andfix-3218.ktregression case. - Replace the lexer's 32-bit action mask with ordered, interned action sequences. Kotlin's
Inside_Identifier -> type(Identifier)command was above index 31 and had been silently omitted, causingfix-3220.ktto fail. The new representation also updates the persisted DFA cache format. Add token-type and parse regression coverage for that input.
Generated target fixes
- Correct Dart driver output paths on Windows: input filenames remain runtime arguments, while an optional
--input-rootpreserves example subdirectories under-o. Update the generated Bash and PowerShell test scripts to pass the root at runtime. Freshly generated Kotlin Dart drivers compiled, andmake testpassed on Windows. - Update the PHP generated app's
phpunit/php-timerdependency from^7.0to^9.0.
Other changes
- Set the tool and package versions to 4.9.0 and remove temporary debugging output.
- Add
proposals/assignment-semantics.md, describing an optional mapping from grammar constructs to language-feature categories, starting with assignment syntax.
Verification
dotnet build src/trparse/trparse.csproj -c Release --no-restorepassed.- The focused Kotlin, lexer-command, and interpreter bundle/DFA-cache suite passed: 26 tests.
- The rebuilt ALL(*) interpreter parsed the formerly failing Kotlin
fix-3220.ktfixture.
See PR #757 for the branch changes.