Repository navigation
Release 4.4.0
Summary
- Bump Trash tools and packages to 4.4.0, including release scripts and the local tool manifest.
- Restore fast DFA-backed parsing for XQuery-hook workloads: parser-only hooks keep eager lexing and the lexer DFA; predicate-independent SLL closures use the parser DFA and fall back to full LL when needed. Parsed hook expressions are cached across files while mutable hook state remains per input.
- Stream the compressed JDK archive directly into the Java AllStar/XQuery example instead of creating an intermediate tar.
- Simplify G4X by removing ANTLR-style element and alternative labels, with corresponding parser, interpreter, compatibility-test, and documentation updates.
- Add an external, ANTLR3-inspired
.astrewrite notation and a post-CST arithmetic prototype, then implementtrparse --allstar --ast-mapto build a pipeableUnvParseTreeNodeAST during parsing rather than constructing a CST first. The normal PAX/tar.ptoutput remains consumable by Trash tree and XQuery commands. - Add a LINVAST-inspired example using action-free C, Java, Kotlin, and Lua grammars with separate mapping files. Expand the mappings to cover representative declarations, functions, classes, expressions, branches, loops, and other language constructs. Extend the mapping language with optional/repeated captures and multi-node text captures, and add checked tree fixtures and tests.
Validation
dotnet test -c Release tests/AllStarParserTests/AllStarParserTests.csproj --no-restore: 571 passed.bash examples/ast-during-parse/run-example.shandbash examples/linvast-ast/run-example.shpassed, including exact AST tree comparisons and XQuery consumption of the resulting parse bundles.- On JDK
Destinations.java(2,719 tokens), the original parser-only hook path took about 225 s. After the DFA/cache changes, a three-file run took 0.40 s for the first hooked file and 0.089/0.069 s for the next two, near the unhooked 0.21/0.086/0.067 s. Full-LL fallbacks with hooks dropped from 4,309 to 277.
Current scope
The AST mapping is a proof of concept, not a complete LINVAST implementation. --ast-map currently targets ordinary pre-tokenized ALL(*) parsing and does not combine with indirect left recursion, context-aware lexing, or runtime XQuery hooks. The Kotlin demo uses a limited ASCII-only lexer because the full upstream Unicode lexer stalls in the current interpreter; its source files retain the original for reference.