Parser does not seem to terminate when parsing this string

[jsyeo]

Environment

My ruby -v says:

jruby 9.1.12.0 (2.3.3) 2017-06-15 33c6439 Java HotSpot(TM) 64-Bit Server VM 25.131-b11 on 1.8.0_131-b11 +jit [darwin-x86_64]

And I am using OS X 10.12.6.

Actual Behavior

When reading a file that contains this comment:

    #     policy: "\"Version\": \"2012-10-17\",\n    \"Id\": \"custom-policy-2016-12-07\",\n    \"Statement\": [\n        {\n            \"Sid\": \"Enable IAM User Permissions\",\n            \"Effect\": \"Allow\",\n            \"Principal\": {\n                \"AWS\": \"arn:aws:iam::111122223333:root\"\n            },\n            \"Action\": \"kms:*\",\n            \"Resource\": \"*\"\n        },\n        {\n            \"Sid\": \"Allow access for Key Administrators\",\n            \"Effect\": \"Allow\",\n            \"Principal\": {\n                \"AWS\": [\n                    \"arn:aws:iam::111122223333:user/ExampleAdminUser\",\n                    \"arn:aws:iam::111122223333:role/ExampleAdminRole\"\n                ]\n            },\n            \"Action\": [\n                \"kms:Create*\",\n                \"kms:Describe*\",\n                \"kms:Enable*\",\n                \"kms:List*\",\n                \"kms:Put*\",\n                \"kms:Update*\",\n               

The jruby interpreter seems to run forever. Even when I have removed everything from the ruby file except for this line, it still seems to not terminate when parsing this particular string.

Here's the stack trace of the non-terminating code when I've tried to dump the running thread:

"main@1" prio=5 tid=0x1 nid=NA runnable
  java.lang.Thread.State: RUNNABLE
	  at org.joni.ByteCodeMachine.matchAt(ByteCodeMachine.java:307)
	  at org.joni.Matcher.matchCheck(Matcher.java:304)
	  at org.joni.Matcher.searchInterruptible(Matcher.java:480)
	  at org.jruby.RubyRegexp$SearchMatchTask.run(RubyRegexp.java:268)
	  at org.jruby.RubyRegexp$SearchMatchTask.run(RubyRegexp.java:249)
	  at org.jruby.RubyThread.executeTask(RubyThread.java:1485)
	  at org.jruby.RubyRegexp.matcherSearch(RubyRegexp.java:232)
	  at org.jruby.lexer.LexingCommon.parseMagicComment(LexingCommon.java:835)
	  at org.jruby.lexer.yacc.RubyLexer.yylex(RubyLexer.java:877)
	  at org.jruby.lexer.yacc.RubyLexer.nextToken(RubyLexer.java:347)
	  at org.jruby.parser.RubyParser.yyparse(RubyParser.java:1617)
	  at org.jruby.parser.RubyParser.yyparse(RubyParser.java:1568)
	  at org.jruby.parser.RubyParser.parse(RubyParser.java:5365)
	  at org.jruby.parser.Parser.parse(Parser.java:128)
	  at org.jruby.parser.Parser.parse(Parser.java:105)
	  at org.jruby.Ruby.parseFileAndGetAST(Ruby.java:2677)
	  at org.jruby.Ruby.parseFileFromMainAndGetAST(Ruby.java:2670)
	  at org.jruby.Ruby.parseFileFromMain(Ruby.java:2658)
	  at org.jruby.Ruby.parseFromMain(Ruby.java:610)
	  at org.jruby.Ruby.runFromMain(Ruby.java:556)
	  at org.jruby.Main.doRunFromMain(Main.java:417)
	  at org.jruby.Main.internalRun(Main.java:305)
	  at org.jruby.Main.run(Main.java:232)
	  at org.jruby.Main.main(Main.java:204)

Hopefully that helps. 😉

Expected Behavior

MRI seems to parse this line properly and terminates immediately.

[enebo]

@jsyeo thanks for reporting this. The lexer processes comment lines looking for magic comment pragmas and we use a regexp. I am guessing something about this string causes our regexp engine to behave pathologically. This definitely helps since it gives a realistic line where we choke.