While I was using YARD I found another bug in Ripper. The following code runs fine with ruby 2.0.0p0 in Windows:
# coding: UTF-8
However, if I run it with JRuby 1.7.10 then I get a cryptic error. In the shell output below you can see the exact version of JRuby I am using and the error:
$ jruby -v && jruby jruby_bug.rb
jruby 1.7.10 (1.9.3p392) 2014-01-09 c4ecd6b on Java HotSpot(TM) 64-Bit Server VM 1.7.0_45-b18 [Windows 8-amd64]
RuntimeError: lex_p < tokp
parse at org/jruby/ext/ripper/RubyRipper.java:306
parse at c:/jruby-1.7.10/lib/ruby/1.9/ripper/core.rb:23
(root) at jruby_bug.rb:3
This is probably a bug in Ripper because eval can happily process the same string that Ripper.parse chokes on.
(this was copied from issue #789)
Fixes #1417. Ripper error with mbc
@DavidEGrayson could you make sure you don't run into any more errors. I am slowly nailing these errors and this new mechanism for handling multibyte chars should fix a whole class of issues.
I think the fixes for this caused CI regression.
yeah I thought I had excluded the single failure (it did not happen for me before I did the commit). I am going to fix it now though (it is non-trivial).
Just double-checked. The failures happening now have nothing to do with ripper? (I did exclude the single failure introduced)
Thanks for fixing this @enebo! It's a bit tricky for me to go reproduce the original problem I saw on January 18th while using YARD, but I can confirm that JRuby 1.7.11 does not exhibit the specific, reduced bug I reported above.
Cool. I suspect you will eventually run into another multibyte char bug but for strings and regexp literals that last fix addressed the majority of cases.