Skip to content
Browse files

Added some info to the README file about the new parser shortcuts. Re…

…moved the item from the TODO file. Added tests for using the list shortcuts with array_args. Fixed the array_args callbacks for the list shortcuts.
  • Loading branch information...
1 parent e351fc0 commit 7fe9f46e308ce62420fc302d6e29c4f7c3d03704 @chriswailes committed Jul 30, 2012
Showing with 82 additions and 24 deletions.
  1. +38 −0 README.md
  2. +0 −4 TODO
  3. +4 −11 lib/rltk/cfg.rb
  4. +3 −3 lib/rltk/parser.rb
  5. +37 −6 test/tc_parser.rb
View
38 README.md
@@ -170,6 +170,44 @@ The default starting symbol of the grammar is the left-hand side of the first pr
**Make sure you call `finalize` at the end of your parser definition, and only call it once.**
+### Shortcuts
+
+RLTK provides several shortcuts for common grammar constructs. Right now these shortcuts include the {RLTK::Parser.empty_list} and {RLTK::Parser.nonempty_list} methods. An empty list is a list that may contain 0, 1, or more elements, with a given token seperating each element. A non-empty list contains **at least** 1 element.
+
+This example shows how these shortcuts may be used to define a list of integers separated by a `:COMMA` token:
+
+ class ListParser < RLTK::Parser
+ nonempty_list(:int_list, ['INT'], :COMMA)
+
+ finalize
+ end
+
+If you wanted to define a list of floats or integers you could define your parser like this:
+
+ class ListParser < RLTK::Parser
+ nonempty_list(:mixed_list, ['INT', 'FLOAT'], :COMMA)
+
+ finalize
+ end
+
+A list may also contain multiple tokens between the separator:
+
+ class ListParser < RLTK::Parser
+ nonempty_list(:foo_bar_list, ['FOO BAR'], :COMMA)
+
+ finalize
+ end
+
+Lastly, you can mix all of these features together:
+
+ class ListParser < RLTK::Parser
+ nonempty_list(:foo_list, ['FOO BAR', 'FOO BAZ+'], :COMMA)
+
+ finalize
+ end
+
+The productions generated by these shortcuts will always evaluate to an array. In the first two examples above the productions will produce a 1-D array containing the values of the `INT` or `FLOAT` tokens. In the last two examples the productions `foo_bar_list` and `foo_list` will produce 2-D arrays where the top level array is composed of tuples coresponding to the values of `FOO`, and `BAR` or one or more `BAZ`s.
+
### Precedence and Associativity
To help you remove ambiguity from your grammars RLTK lets you assign precedence and associativity information to terminal symbols. Productions then get assigned precedence and associativity based on either the last terminal symbol on the right-hand side of the production, or an optional parameter to the {RLTK::Parser.production} or {RLTK::Parser.clause} methods. When an {RLTK::Parser} encounters a shift/reduce error it will attempt to resolve it using the following rules:
View
4 TODO
@@ -1,9 +1,5 @@
parser.rb:
Composable parsers.
- Add shortcuts for common productions like:
- * Empty lists
- * Non-empty lists
-
Auto-generate as much of the AST as possible, including AST node classes.
Add a simple way to lexically scope the language.
View
15 lib/rltk/cfg.rb
@@ -144,17 +144,10 @@ def clause(expression)
rhs <<
case ttype1
- when :'?'
- self.get_question(tvalue0)
-
- when :*
- self.get_star(tvalue0)
-
- when :+
- self.get_plus(tvalue0)
-
- else
- tvalue0
+ when :'?' then self.get_question(tvalue0)
+ when :* then self.get_star(tvalue0)
+ when :+ then self.get_plus(tvalue0)
+ else tvalue0
end
else
rhs << tvalue0
View
6 lib/rltk/parser.rb
@@ -207,9 +207,9 @@ def array_args
when :nelp
case num
- when :first then Proc.new { |v| v }
- when :second then Proc.new { |v| v[0] }
- else Proc.new { |v| v[0] << v[2] }
+ when :first then Proc.new { |v| v }
+ when :second then Proc.new { |v| v[0] + [v[2]] }
+ else Proc.new { |v| if v.length == 1 then v.first else v end }
end
end,
p.rhs.length
View
43 test/tc_parser.rb
@@ -32,10 +32,7 @@ class ABLexer < RLTK::Lexer
end
class AlphaLexer < RLTK::Lexer
- rule(/a/) { |t| [t.upcase.to_sym, t] }
- rule(/b/) { |t| [t.upcase.to_sym, t] }
- rule(/c/) { |t| [t.upcase.to_sym, t] }
- rule(/d/) { |t| [t.upcase.to_sym, t] }
+ rule(/[A-Za-z]/) { |t| [t.upcase.to_sym, t] }
rule(/,/) { :COMMA }
@@ -88,12 +85,20 @@ class ArrayCalc < RLTK::Parser
finalize
end
- class EmptyListParser < RLTK::Parser
+ class EmptyListParser0 < RLTK::Parser
empty_list('list', ['A'], :COMMA)
finalize
end
+ class EmptyListParser1 < RLTK::Parser
+ array_args
+
+ empty_list('list', ['A', 'B', 'C D'], :COMMA)
+
+ finalize
+ end
+
class NonEmptyListParser0 < RLTK::Parser
nonempty_list('list', ['A'], :COMMA)
@@ -112,6 +117,12 @@ class NonEmptyListParser2 < RLTK::Parser
finalize
end
+ class NonEmptyListParser3 < RLTK::Parser
+ nonempty_list('list', ['A+'], :COMMA)
+
+ finalize
+ end
+
class DummyError1 < StandardError; end
class DummyError2 < StandardError; end
@@ -229,8 +240,20 @@ def test_ebnf_parsing
end
def test_empty_list
+ ####################
+ # EmptyListParser0 #
+ ####################
+
expected = []
- actual = EmptyListParser.parse(AlphaLexer.lex(''))
+ actual = EmptyListParser0.parse(AlphaLexer.lex(''))
+ assert_equal(expected, actual)
+
+ ####################
+ # EmptyListParser1 #
+ ####################
+
+ expected = ['a', 'b', ['c', 'd']]
+ actual = EmptyListParser1.parse(AlphaLexer.lex('a, b, c d'))
assert_equal(expected, actual)
end
@@ -351,6 +374,14 @@ def test_nonempty_list
assert_raise(RLTK::NotInLanguage) { NonEmptyListParser2.parse(AlphaLexer.lex('c')) }
assert_raise(RLTK::NotInLanguage) { NonEmptyListParser2.parse(AlphaLexer.lex('d')) }
+
+ #######################
+ # NonEmptyListParser3 #
+ #######################
+
+ expected = [['a'], ['a', 'a'], ['a', 'a', 'a']]
+ actual = NonEmptyListParser3.parse(AlphaLexer.lex('a, aa, aaa'))
+ assert_equal(expected, actual)
end
def test_postfix_calc

0 comments on commit 7fe9f46

Please sign in to comment.
Something went wrong with that request. Please try again.