`python -m pegen python` is broken

[sobolevn]

If I run PYTHONPATH=Tools/peg_generator ./python.exe -m pegen python Grammar/python.gram on main branch, the result file is broken.

There are lots of invalid syntax construct that come from C directly.

Some examples:

    @memoize
    def interactive(self) -> Optional[mod_ty]:
        # interactive: statement_newline
        mark = self._mark()
        if (
            (a := self.statement_newline())
        ):
            return _PyAST_Interactive ( a , p -> arena )
        self._reset(mark)
        return None

It also has this at the bottom:

        return children

    KEYWORDS = ('return', 'import', 'from', 'raise', 'pass', 'del', 'yield', 'assert', 'break', 'continue', 'global', 'nonlocal', 'def', 'if', 'class', 'with', 'for', 'try', 'while', 'as', 'elif', 'else', 'in', 'except', 'finally', 'None', 'True', 'False', 'or', 'and', 'not', 'is', 'lambda')
    SOFT_KEYWORDS = ('match', 'case', '_')


void *
_PyPegen_parse(Parser *p)
{
    // Initialize keywords
    p->keywords = reserved_keywords;
    p->n_keyword_lists = n_keyword_lists;
    p->soft_keywords = soft_keywords;

    // Run parser
    void *result = NULL;
    if (p->start_rule == Py_file_input) {
        result = file_rule(p);
    } else if (p->start_rule == Py_single_input) {
        result = interactive_rule(p);
    } else if (p->start_rule == Py_eval_input) {
        result = eval_rule(p);
    } else if (p->start_rule == Py_func_type_input) {
        result = func_type_rule(p);
    } else if (p->start_rule == Py_fstring_input) {
        result = fstring_rule(p);
    }

    return result;
}

I am attaching full file to this issue: https://gist.github.com/sobolevn/37cfd12ad9d2c157e72e7589747f4ed2

Pinging @isidentical and @pablogsal

[pablogsal]

The grammar in python.gram is a grammar for the C backend. There is nothing broken here: you just cannot use the Python backend with this grammar.

Pegen is executed on every PR to check that the parser is updated so we have checks to know is not broken.

If you want a grammar that works with the python backend, check the 3rd party pegen module:

https://github.com/we-like-parsers/pegen

[pablogsal]

Closing as not a bug.

[sobolevn]

Thanks!