Skip to content

v1.29.0

Latest

Choose a tag to compare

@yhirose yhirose released this 27 Oct 11:37
· 5 commits to master since this release

This release lets a grammar with usr rules use a serialized blob, makes a failed usr show up in syntax errors, and changes what \x and octal escapes mean in a literal. Blobs of earlier versions are no longer read and the meaning of \x changes, so it is a minor release.

✨ New: rules given as usr can be serialized

A rule whose whole body is a usr operator holds code, so a blob now keeps only its name, and the application gives the body again when it loads the blob. A grammar with such a rule no longer has to skip serialize_grammar() and pay for the meta-parse at every start.

peg::Rules rules{{"Name", peg::usr(match)}};
peg::parser parser(text, rules);
auto blob = parser.serialize_grammar();   // keeps the name "Name", not the function

peg::parser parser2;
parser2.load_blob(blob, rules);           // takes the body of "Name" from rules
  • load_blob(blob) without rules still works. It returns false for a blob that has such rules.
  • load_blob(blob, rules) returns false when rules lacks a name the blob needs, or gives it as something other than a usr operator. Names in rules that the blob does not need are ignored.
  • A usr operator anywhere else in a rule body still makes serialize_grammar() throw.
  • load_blob now tells the logger set with set_logger why it failed, at line 0 and column 0.
  • peglint --blob --external NAME writes such a blob. The option may be given more than once.

✨ Changed: a failed usr is reported in syntax errors

A usr function that returned peg::kFail left no trace in the parse's error state, so a parse that failed there told the logger nothing, and in a choice the report listed only the other alternatives. It is now reported like any other failure:

1:7: syntax error, unexpected 'x', expecting <NAME>.

Whether a parse succeeds and the tree it builds do not change. Applications that use usr will see new messages where there were none, and syntax error, ... expecting ... where there was expected end of input.

⚠️ Changes to be aware of

  • Blobs of earlier versions are not read. The version of the blob is now PEG5, and a reader rejects flag bits it does not know. Serialize your grammars again.
  • \x and octal escapes name a byte in a literal (#347). '\xe3\x81\x82' used to match the UTF-8 of U+00E3 U+0081 U+0082 instead of the three bytes, because each escape was encoded as a code point. It now matches the three bytes, as in PCRE, Python bytes and LPeg. In a character class they stay code points, and \u is unchanged.
  • peg::resolve_escape_sequence has a new parameter, bool as_bytes = true. Callers compile unchanged, and programs linked against an earlier version must be rebuilt (ABI).

🔧 Other

  • The GCC -Wattributes warning on the noinline member definitions is gone, and the warning is an error in the build now (#346).
  • The MSVC C4267 warning in test_integration.cc is gone.
  • New tests cover rules given on load, the failure reports of usr, and the reading of flags in a blob.