Missing ambiguities when using inline rules

[yurymann]

Describe the bug

With some grammars, Earley parser does not return all ambiguities with ambiguity='explicit'.
However, when removing the inlining underscore, all ambiguities are returned.

Potentially related to #536.

To Reproduce

start: _field+

_field: f1 | f2 | f3

f1: INT
f2: INT "M"?
f3: INT "M"

%ignore WS
%import common.WS
%import common.INT

When parsing string 1M 2 with ambiguity='explicit', the result misses ambiguities f2 f2 and f3 f1.

>> t = parser.parse('1M 2')
>> print(t.pretty())

_ambig
  start
    f2	1
    f1	2
  start
    f3	1
    f2	2

However, removing _ from field fixes it, at the expense of the additional tree level which I wanted to avoid.

start: field+

field: f1 | f2 | f3

f1: INT
f2: INT "M"?
f3: INT "M"

%ignore WS
%import common.WS
%import common.INT

>> t = parser.parse('1M 2')
>> print(t.pretty())

start
  _ambig
    field
      f2	1
    field
      f3	1
  _ambig
    field
      f1	2
    field
      f2	2

It's great that the workaround works, at least in this case. But with a complex grammar including inlined rules at several levels, it's difficult to spot the issue before realising that some results are missing in production.

[erezsh]

@chanicpanic Any ideas?

[chanicpanic]

Both grammars produce the same SPPF, so the issue is probably in the forest to tree transformation. I will look into it more when I have a chance.

[erezsh]

Makes sense! Possibly the inlining of the rules messes with the logic somehow.

[yurymann]

Many thanks for fixing it so quickly! When do you expect to release it?

[erezsh]

@yurymann Released 1.1.5 today.