Can't generate parser using Rubinius

[jimsynz]

When attempting to run the test suite against Rubinius I get the following backtrace:

An exception occurred running /Users/jnh/.rvm/rubies/rbx-head-rbx19/bin/testrb
    undefined method `id' on nil:NilClass. (NoMethodError)

Backtrace:
                                         Kernel(NilClass)#id (method_missing) at kernel/delta/kernel.rb:81
                                            { } in RLTK::Parser.grammar_prime at lib/rltk/parser.rb:689
                                                                   Array#each at kernel/bootstrap/array.rb:68
                                            { } in RLTK::Parser.grammar_prime at lib/rltk/parser.rb:686
                                                                   Array#each at kernel/bootstrap/array.rb:68
                                            { } in RLTK::Parser.grammar_prime at lib/rltk/parser.rb:681
                                              { } in RLTK::Parser::State#each at lib/rltk/parser.rb:1505
                                                                   Array#each at kernel/bootstrap/array.rb:68
                                                     RLTK::Parser::State#each at lib/rltk/parser.rb:1505
                                            { } in RLTK::Parser.grammar_prime at lib/rltk/parser.rb:676
                                                                   Array#each at kernel/bootstrap/array.rb:68
                                                   RLTK::Parser.grammar_prime at lib/rltk/parser.rb:675
                                                    { } in RLTK::Parser.prune at lib/rltk/parser.rb:1076
                                                                   Array#each at kernel/bootstrap/array.rb:68
                                                    { } in RLTK::Parser.prune at lib/rltk/parser.rb:1074
                                                                   Array#each at kernel/bootstrap/array.rb:68
                                                    { } in RLTK::Parser.prune at lib/rltk/parser.rb:1068
                                                                   Array#each at kernel/bootstrap/array.rb:68
                                                           RLTK::Parser.prune at lib/rltk/parser.rb:1058
                                                        RLTK::Parser.finalize at lib/rltk/parser.rb:615
             ParserTester::EmptyListParser0.__class_init__ (EmptyListParser0) at test/tc_parser.rb:111
                                   ParserTester.__class_init__ (ParserTester) at test/tc_parser.rb:108
                                                            Object#__script__ at test/tc_parser.rb:26
                                                 Rubinius::CodeLoader.require at kernel/common/codeloader.rb:212
                                                       Kernel(Object)#require at kernel/common/kernel.rb:649
                                                            Object#__script__ at test/ts_rltk.rb:34
                                                 Rubinius::CodeLoader.require at kernel/common/codeloader.rb:212
                               Kernel(Test::Unit::AutoRunner::Runner)#require at kernel/common/kernel.rb:649
  { } in Test::Unit::RequireFiles(Test::Unit::AutoRunner::Runner)#non_options at /Users/jnh/.rvm/rubies/rbx-head-rbx19/lib/19/test/unit.rb:221
                                                                   Array#each at kernel/bootstrap/array.rb:68
         Test::Unit::RequireFiles(Test::Unit::AutoRunner::Runner)#non_options at /Users/jnh/.rvm/rubies/rbx-head-rbx19/lib/19/test/unit.rb:215
             Test::Unit::Options(Test::Unit::AutoRunner::Runner)#process_args at /Users/jnh/.rvm/rubies/rbx-head-rbx19/lib/19/test/unit.rb:52
                                          Test::Unit::AutoRunner#process_args at /Users/jnh/.rvm/rubies/rbx-head-rbx19/lib/19/test/unit.rb:625
                                                            Object#__script__ at /Users/jnh/.rvm/rubies/rbx-head-rbx19/bin/testrb:5
                                             Rubinius::CodeLoader#load_script at kernel/delta/codeloader.rb:68
                                             Rubinius::CodeLoader.load_script at kernel/delta/codeloader.rb:118
                                                      Rubinius::Loader#script at kernel/loader.rb:615
                                                        Rubinius::Loader#main at kernel/loader.rb:816
Coverage report generated for Unit Tests to /Users/jnh/Dev/forks/RLTK/coverage. 699 / 763 LOC (91.61%) covered.
rake aborted!
Command failed with status (1): [/Users/jnh/.rvm/rubies/rbx-head-rbx19/bin/...]

Tasks: TOP => test
(See full trace by running task with --trace)

The backtrace points at lib/rltk/parser.rb:689 as being the culprit:

      def grammar_prime
        if not @grammar_prime
          @grammar_prime = CFG.new

          @states.each do |state|
            state.each do |item|
              lhs = "#{state.id}_#{item.next_symbol}".to_sym

              next unless CFG::is_nonterminal?(item.next_symbol) and not @grammar_prime.productions.keys.include?(lhs)

              @grammar.productions[item.next_symbol].each do |production|
                rhs = ""

                cstate = state

                production.rhs.each do |symbol|
                  rhs += "#{cstate.id}_#{symbol} "

                  cstate = @states[cstate.on?(symbol).first.id]  # <-- here
                end

                @grammar_prime.production(lhs, rhs)
              end
            end
          end
        end

        @grammar_prime
      end

The parser code is too complex for me to figure out at first blush, so I thought I'd raise an issue in the hopes that you had any ideas/pointers on where I should look?

[chriswailes]

It appears that a nil value is somehow being passed into the State#on method, which causes State#on? to return an array with the nil value in it. I don't have time to look at this, and to be honest it is pretty low priority as this is for Rubinius, so I would suggest that if you want to track this down yourself you should try instrumenting the State#on method.

[jimsynz]

Thanks. Will do.

[chriswailes]

OK, this has now been fixed (thanks to @benedikt) and will be part of RLTK 3.0, which I will release in a couple of days.