Skip to content

v1.28.0

Choose a tag to compare

@yhirose yhirose released this 25 Oct 22:24
· 16 commits to master since this release

This release adds a form of usr that does not stop the parser's optimizations, and fixes the user data that a recovery expression received. It has a source-incompatible change to usr and an ABI change, so it is a minor release.

✨ New: usr for the input alone

usr also takes a function of the input, (const char *s, size_t n), or of the input and the user data, (const char *s, size_t n, std::any &dt). It returns the length it matches, or peg::kFail.

peg::parser parser(grammar, {{"TaggedString", peg::usr([](const char *s, size_t n) -> size_t {
  // read a string whose end the grammar cannot express
})}});

A usr given the semantic values ((s, n, vs, dt)) can read and change them, so a grammar that has one builds every value and records no AST. A usr of the input alone cannot, so the grammar keeps skipping the values nobody reads and keeps recording the AST. On the DuckDB grammar with big.sql, one usr of the first kind took 4,905M instructions per parse (with packrat parsing and enable_ast(true), built with -O3 -DNDEBUG, counted with proc_pid_rusage around one parse), and one of the second kind took 2,375M, the same as without the usr.

Put a lookahead such as &'$' in front of the rule: a usr can start with any character, so without it the parser tries the function wherever the rule is tried.

🐛 Fixed: user data in a recovery expression

The expression of a recovery (%recover) was parsed with an empty std::any instead of the dt of the parse, so a usr or an action in it saw no user data, and reading it could throw std::bad_any_cast. It now gets the dt of the parse. The rules under %word still get no user data, because whether a literal is a word is decided once and kept across parses; the README says so.

⚠️ Changes to be aware of

  • &peg::usr no longer names one function, because usr is overloaded. Use a lambda that calls it if you need a pointer. Calls that pass a function, as before, compile unchanged: everything the old usr accepted keeps its meaning.
  • peg::User has a new member, so code compiled against an earlier version must be rebuilt (ABI).

🔧 Other

  • The unused parameter warning of a release build in Holder::parse_recognized is gone.
  • benchmark/duckdb compares cpp-peglib with DuckDB's parsers on one input. just duckdb-bench fetches the DuckDB libraries, builds and runs it (macOS). just clean removes the build directories and the fetched files.
  • New tests cover the usr forms, the reading of values by a usr, and the user data of a recovery expression.