More infinite recursion

[MichaelClerx]

We're seeing a similar issue to #85 when using pyparsing 2.4.2, that doesn't occur with 2.3.1.

Selected part of the testing output below:

../../../virtualenv/python3.6.3/lib/python3.6/site-packages/fc/parsing/CompactSyntaxParser.py:1577: in CompactSyntaxParser
    expr.ignore('\\' + p.LineEnd())
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:4450: in ignore
    self.expr.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:4446: in ignore
    self.expr.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:4446: in ignore
    self.expr.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:3880: in ignore
    e.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:3880: in ignore
    e.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:4446: in ignore
    self.expr.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:3880: in ignore
    e.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:3880: in ignore
    e.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:4446: in ignore
    self.expr.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:3880: in ignore
    e.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:3880: in ignore
    e.ignore(self.ignoreExprs[-1])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/pyparsing.py:4446: in ignore
    self.expr.ignore(self.ignoreExprs[-1])

...

E   RecursionError: maximum recursion depth exceeded while calling a Python object

[ptmcg]

I'm having trouble reproducing this - can you point me to some online source containing your parser code?

[MichaelClerx]

Thanks! It's in a fairly complex bit of code I'm afraid

https://github.com/ModellingWebLab/weblab-fc/blob/pep8-names/fc/parsing/CompactSyntaxParser.py#L1420-L1577

The last line in that snippet is the one that triggers it

[ptmcg]

There is a bug around line 2565 of pyparsing.py:

        if PY_3:
            self is other or super(ParserElement, self).__eq__(other)

should read:

        if PY_3:
            return self is other or super(ParserElement, self).__eq__(other)

I think this may affect how ignore handles duplicate expressions when it is called more than once, or in a recursive grammar.

Also, is there any way you can just call ignore at the topmost level of your grammar (protocol?) instead of at the intermediate expr or simpleExpr levels? This is the real intention in ignore, is to do so at the topmost level, and then have ignore recurse through its own structure to apply the ignorable expression.

[MichaelClerx]

Thanks!
I'm not sure I understand the __eq__ function's implementation for Python3:

def __eq__(self, other):
    if isinstance(other, ParserElement):
        if PY_3:
            self is other or super(ParserElement, self).__eq__(other)
        else:
            return self is other or vars(self) == vars(other)
    elif isinstance(other, basestring):
        return self.matches(other)
    else:
        return super(ParserElement, self) == other

• The self is other check could be at the very top (if self is other return True), and surely doesn't need an isinstance check first?
• The alternative check is super(ParseElement, self).__eq__(other), which calls the eq function of the object class, which, as far as I can tell, does the same as self is other
• If you move the self is other check to the top, both the first branch (isinstance(other, ParserElement)) and the last (else) call eq on the parent object class

Not sure why the vars approach is dropped in Python3 either? I can avoid the resolve my issue by rewriting e.g. like this:

    def __eq__(self, other):
        if self is other:
            return True
        elif isinstance(other, basestring):
            return self.matches(other)
        elif isinstance(other, ParserElement):
            return vars(self) == vars(other)
        return False

[ptmcg]

This looks Py2-Py3 compatible, so I could drop this in a part of a 2.4.3 release, I think. Thanks for testing it out.

[ptmcg]

When you say "I can avoid the issue", does that mean that this change resolves your recursion problem?

[MichaelClerx]

When you say "I can avoid the issue", does that mean that this change resolves your recursion problem?

It does, yes!

[ptmcg]

Fixed in 2.4.3, released today