Improve support for new front-end

[rengolin]

With the new front-end (#111, #116) merged, we need to make sure the existing examples parse correctly into an acceptable AST shape, that we can later lower into MLIR (#117). However, these tasks are not dependent on each other, as only enough of the AST is needed to convert, while this task is to extend that support.

There are already a few tests that work:

./resolution/compile-pass/circular.verona
./resolution/compile-pass/interleaved.verona
./reachability/compile-pass/visit-variable-renaming-type.verona
./reachability/compile-pass/visit-receiver.verona
./features/compile-pass/library/sibling.verona
./features/compile-pass/cell.verona
./typechecking/compile-pass/field-write.verona

while others emit errors when trying to parse a class declaration.

The idea is to discuss, from the semantics, what syntax we want, and change either parser or examples to fit the consensus.

Acceptance criteria:

• Existing tests covered or explicitly ignored / removed
• Changes to the parser, grammar, examples to suit the consensus semantics
• Tests that regexp check the AST format from the output through examples

Optional:

• New examples that don't necessarily parse with the old pegmatite front-end

Not included:

• Lowering to MLIR, though there will be collaboration with feedback from what's easier for Convert new AST into MLIR #117

[yhirose]

Hello! I am the author of cpp-peglib, and glad to find you get to try it out.

I'll give you some background of this library for your information, since this PEG library is somewhat unique compared to other PEG libraries.

As you may have already noticed, this library is good for parser prototyping, since its grammar format is just plain text and we can develop grammar by trail and error with lint tools (peglint command line tool and playground web application) without compiling.

cpp-peglib also has simple built-in AST generation support. The default AST node can be enhanced easily with your own data. Although it produces a pretty redundant AST tree as default, you can remove unnecessary nodes with ignore operator ~ and AstOptimizer to make it good shape.

Another feature that cpp-peglib provides is infix expression support. This is not a part of the original PEG spec, but it's pretty useful to handle infix expression with many precedence levels.

Of course, a parser generated from cpp-peglib cannot be any better than hand crafted parsers in many respects. But I believe it can at least save your time during early stage of language design until the language syntax settles down, then shift to a product version of a hand written parser for performance and good error handling.

Anyway, thanks for your interest in cpp-peglib, and hope it helps your project!

P.S. I apologize if this comment shouldn't have been posted here.

[rengolin]

@yhirose, thanks for the feedback! Most welcome.

Our use of cpp-peglib is exactly as you describe. We're writing a prototype, to quickly build an AST that we can use from an evolving language. Once it becomes more stable, we'll have to write a custom parser, but hopefully retaining the AST mostly intact, so that we don't impact the other parts of the compiler.

[rengolin]

After a long time, most of the old syntax is changing and we have enough of the new syntax in the MLIR generator that we can mark this one "done". If we find missing parts in the language support (of which there are many), we can create new issues on each one specifically.