v1.17.0

⚡ Performance

Recognition-only parses skip the semantic-value machinery entirely

When parse() runs with no actions, enter/leave callbacks, predicates, User opes, PrecedenceClimbing, tracers, or error reporting attached, none of them can ever observe a semantic value. The parser now detects this once per parse and enters a recognizer mode: rule invocations parse straight into the caller's scope with packrat memoization and re-entry guards intact, and TokenBoundary skips token capture. Left-recursive rules keep the full seed-growing machinery unchanged, and any parse with a callback attached — including AST construction — takes the same general path as before.

This is the release's biggest win, and it landed alongside five smaller cuts to per-invocation bookkeeping that were only worth making once the semantic-value overhead was out of the way:

• Holder::parse_core writes the parse result straight into the caller's slot instead of moving it through a pair<size_t, any>, and only maintains the rule stack when something will read it (needs_rule_stack for error reporting/tracers, has_macro_ref for rules containing a macro call).
• SemanticValues now points at the owning Definition's name instead of copying it into a std::string on every match.
• PrioritizedChoice skips its cut_stack bookkeeping entirely in grammars with no Cut or Recovery ope.
• Reference skips trace-state save/restore without a tracer attached, and skips pushing an argument-scope frame for callees nothing reads it.
• Argument-scope frames (args_stack) are now reused across calls the way value_stack already was, instead of being reallocated per macro invocation.
• Memoized match lengths for the (small, selectively-chosen) packrat set live in a flat array indexed directly, instead of going through the open-addressing hash on every read and write.

Measured on the SQL benchmark (Apple M2 Max, release build, big.sql ~1.2MB, A/B alternating rounds), against the benchmark's own YACC baseline (pg_query_parse(), which also JSON-serializes the tree — see the methodology note below for what that baseline actually measures): recognition-only parsing went from roughly parity with that baseline down to consistently faster across the whole sequence of changes.

AST construction: -14% (still the dominant cost for tree-building parses)

Two changes cut the cost of enable_ast() parses:

• Child vectors are moved, not copied. SemanticValues::transform() now reserves before growing, and add_ast_action builds each node with an empty child vector, then move-assigns the collected children into it — stealing the buffer instead of copying it and bumping every child's reference count. AstBase's constructors are unchanged, so custom annotation types keep compiling as-is.
• line_info() remembers the line it last found. AST construction asks for the line/column of every node, and each lookup was a binary search over every line start in the input (on big.sql, about five million probes into a 295KB table). Callers walk the input in bursts, so the previous answer is usually still right; line_info() now checks it first with two comparisons and only falls back to the binary search when the position has moved off it.

Measured the same way: big.sql PEG-ast/YACC went from 3.7-3.8x to 3.16-3.35x, an average of -14% across both changes. Node equivalence was checked field-by-field (name, tag, line, column, position, length, choice, parent) across all 330,733 nodes of big.sql before and after — identical.

📏 Benchmark methodology correction

The benchmark's YACC row calls pg_query_parse(), which — on top of parsing and building the tree — stringifies the entire result to JSON (6.9MB for big.sql). Neither PEG row does anything like that, so every ratio against it was mixing "parse" with "parse plus serialize" on one side only. Isolated with pg_query_split_with_parser() (same pg_query_raw_parse(), no JSON step — verified against the 6.2.2 source), JSON serialization turns out to be close to 60% of what the old row measured.

Against the tree-only baseline, PEG recognition is 1.9x slower than YACC's parse+tree (not the ~0.8x the JSON-inclusive ratio suggested), and PEG-ast is 7.8x slower (not ~3.3x). The benchmark now prints both: a new YACC-parse: big.sql row plus PEG/YACC-parse and PEG-ast/YACC-parse ratio lines that are like-for-like, alongside the original YACC/PEG/YACC/PEG-ast/YACC lines kept and labeled as not like-for-like for continuity.

The benchmark's PEG-ast: big.sql row (full AST construction, not just recognition) was also added this release, making the tree-building cost visible for the first time instead of only the recognition-only number.

✨ New Features

• peglint --version prints the library version (CPPPEGLIB_VERSION from peglib.h) and exits.

🐛 Bug Fixes

Recognizer mode's keyword-guard fast path leaked tokens

Sequence::parse_core's keyword guard fuses a token boundary into itself and emits the matched token directly, bypassing TokenBoundary::parse_core. Recognizer mode's token-capture guard was added to the latter but missed this copy, and since recognizer mode no longer pushes a per-rule semantic-value scope, the tokens accumulated in the parse's top-level scope for the whole parse. Parsing big.sql with no actions attached retained 51,642 string_views (~826KB) that nothing ever read; a single attached action (which turns recognizer mode off) left the count at zero. Fixed by applying the same guard in the keyword-guard path.

🧹 Internal / Cleanup

• Context::recognizer renamed to recognize_only, matching the predicate naming (has_tracer, has_cut, is_macro, …) used everywhere else nearby.
• The recognizer fast path and the general parse body had duplicated the dozen lines that maintain the rule stack and honor no_whitespace, differing only in which semantic-value scope they parsed into; both now call a shared Holder::parse_ope_body.

⚠️ Compatibility Notes

• As a header-only library, this only requires recompiling — there is no shared object to relink against. But Context, Definition, AssignIDToDefinition, and the packrat cache all gained or renamed internal state this release (recognizer-mode tracking, has_cut/has_opaque_ope, a dense match-length array replacing part of the packrat hash), so don't mix object files built against different peglib.h versions in the same binary.
• No public API changes. All 520 tests plus the 43-test spec harness pass throughout.

Full Changelog: v1.16.0...v1.17.0