v1.24.0

This release fixes a performance regression of v1.23.0 and makes loading a grammar blob more robust. What a grammar matches and what errors it reports do not change.

⚠️ Changes to be aware of

API and ABI

• FirstSet::can_be_empty is renamed FirstSet::matches_empty_on_other_bytes. It tells whether an expression matches empty on a byte it cannot start with, which for & differs from whether it can match empty: &'a' can match empty, but fails on any byte but 'a'. Definition::can_be_empty still tells whether a rule can match empty.
• ComputeFirstSet no longer overrides visit(AndPredicate&), which changes its vtable. peg::work_out_can_be_empty(Grammar&) is new.

Tracing and callbacks

An alternative that starts with & is now skipped where the lookahead's operand cannot start, as other alternatives are. A trace shows fewer tries of such alternatives, and the enter and leave actions of the rules in them do not run where they are skipped.

⚡ Performance

v1.23.0 tried every alternative and rule that starts with &, on every byte, with or without a logger. They are now skipped where the lookahead's operand cannot start, as LPeg does.

Instructions of one parse against v1.23.0. The large grammar is the grammar of DuckDB's PEG parser ported to cpp-peglib (about 1,100 rules), on a 1.2MB SQL file with packrat parsing; its operator rule starts with &.

• Large grammar, recognition: 3096M -> 2907M (-6.1%), 3.3% fewer than v1.22.0 (3006M).
• With a logger: 3744M -> 3516M (-6.1%). With set_max_depth: -6.2%.
• enable_ast(true): 5161M -> 4925M (-4.6%). enable_ast(): -2.4%. Without packrat parsing: -5.1%.
• benchmark/data/sql.peg, the left-recursive grammars and the README's macro example are unchanged.
• Where & starts with a rule that gives a message, a parse with a logger checks the alternatives of every choice, as it already does where an alternative starts with such a rule: +4.2% on a small keyword grammar of that shape.

🐛 Fixes

• Grammar blobs: whether a rule can match empty is now worked out again when a blob is loaded, instead of read from it. A hand-made or damaged blob saying that a rule cannot match empty when it can had a rule skipped and an action not run: with S <- P? 'q', P <- E 'p' and E <- 'e'?, E's action ran once on q with the grammar loaded from text and not at all from such a blob. The format is unchanged: blobs written by v1.23.0 load in v1.24.0 and the other way round. load_blob() takes 3% to 6% more instructions.

🔧 Internals

• Which rules can match empty is worked out by one function for grammars loaded from text and from blobs and for the meta grammar, for which it was never worked out. An assert checks each rule's first set against it.