Added support for Left Recursive Grammars, using the algorithm from the Ometa paper #19

Open
wants to merge 9 commits into
from

Conversation

Projects
None yet
4 participants
Contributor

joachimm commented Feb 22, 2011

Also added some stuff in the TextMate bundle

Owner

mjackson commented Feb 22, 2011

Wow! This is awesome. Thanks for this work. I'll be reviewing these commits as soon as I get the chance.

Is http://www.vpri.org/pdf/tr2007002_packrat.pdf [1] the Ometa paper? I didn't know what ometa was until I started searching for it. The reason I ask is because I'm reading through http://tratt.net/laurie/research/publications/papers/tratt__direct_left_recursive_parsing_expression_grammars.pdf [2] and the author, Tratt, makes the observation that Warth's Ometa tool, which implements the technique described in [1], incorrectly parses the expression 1-2-3, as (1-(2-(3))), given the grammar:
expr <- expr "-" expr / num
num <- [0-9]+
joachimm, does your implementation of Warth's technique produce the same right-associative parse as Warth's Ometa tool? I'm thinking that Tratt's alteration to Warth's algorithm might parse a broader class of languages. On the other hand, this might not be an issue at all. I'm just bringing it up because I haven't tinkered around enough with your implementation to know, and reading through Tratt's paper brought the issue to my attention.

Thanks!

Owner

mjackson commented May 13, 2011

David, Thanks for researching this. As I mentioned above, I'd love to be able to support left recursion in Citrus, but I haven't had the time to thoroughly test the algorithm myself yet. It's an important change, and it deserves careful consideration.

Contributor

joachimm commented May 14, 2011

David, yes [1] is the Ometa paper. According to this test I believe left associtivity works as intended https://github.com/joachimm/citrus/blob/master/test/indirect_left_recursion_test.rb

joachimm closed this May 14, 2011

joachimm reopened this May 14, 2011

I just tried implementing the special case (a grammar production is both left-recursive and right-recursive) that Tratt documented in his paper [2].

The grammar is:
expr <- expr "-" expr / num
num <- [0-9]+

In left_recursion_test.rb, I slightly modified your LR grammar:

  grammar :LR do
    rule :expr do
      any(all(label(:expr, 'lhs'), '-', label(:expr, 'rhs')){lhs.value - rhs.value},
          :num)
    end

    rule :num do
      ext(/[0-9]+/){to_i}
    end
  end

  def test_lr
    match = LR.parse("3-4-5", {:leftrecursive=>true})
    assert(match)
    assert_equal("3-4-5", match)
    assert_equal(-6, match.value)
  end

Unless I've done something wrong, which is not unlikely, this test fails because 3-4-5 is parsed as (3 - (4 - 5)) and produces the result 4.

Can you verify that this is in fact an issue. I'm not 100% sure that I've set up a proper test to verify whether or not Tratt's observation is an issue in this implementation.

Thanks.

Contributor

joachimm commented May 14, 2011

You are right. Making rules both left-recursive and right-recursive, seems to give unwanted results. Thanks for the link to the Tratt's paper. I will see if I can fit his idea into the algorithm.

Hey @joachimm have you made any progress on this? I'm interested in whether Warth's implementation is correct?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment