Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

java.lang.ClassCastException: org.jruby.RubyNil cannot be cast to org.jruby.RubyMatchData #6640

Closed
gerases opened this issue Mar 30, 2021 · 8 comments · Fixed by #6644
Closed

Comments

@gerases
Copy link

gerases commented Mar 30, 2021

Environment Information

Jruby version: jruby 9.2.17.0 (2.5.8) 2021-03-29 84d363d OpenJDK 64-Bit Server VM 25.282-b08 on 1.8.0_282-b08 +jit [linux-x86_64]

Operating system: Linux 9738431d2f6d 3.10.0-1160.11.1.el7.x86_64 #1 SMP Fri Dec 18 16:34:56 UTC 2020 x86_64 GNU/Linux

Expected Behavior

The skeleton code is:

  class TEST_CLASS
    # Class method
    def self.run
      threads = []
      some_array.each do |input|
        threads << Thread.new do
          obj = TEST_CLASS.new(input)
          obj.process do |line|
            yield line
          end
        end
      end
    end

    # Instance methods
    def initialize(input)
      @input = input
    end

    # Prints a line unless the block returns false
    def process
      open(input).each do |line|
        next unless yield line
        puts line
      end
    end
  end

  TEST_CLASS.run do |line|
    raw_date=line.split(/\s+/)[5..6].join(' ')
    Date.parse(raw_date) <= Date.parse('2021-01-01')
  end

When running that code under jruby, I expect the code to finish without exceptions

Actual Behavior

When running the above code under jruby, I get the following exception:

java.lang.ClassCastException: org.jruby.RubyNil cannot be cast to org.jruby.RubyMatchData

There are two things that fix the problem:

  • Removing the call to split
  • Moving the block inside TEST_CLASS like so:
    ...
    def self.run
      threads = []
      some_array.each do |input|
        threads << Thread.new do
          obj = TEST_CLASS.new(input)
          obj.process do |line|
             raw_date=line.split(/\s+/)[5..6].join(' ')
             Date.parse(raw_date) <= Date.parse('2021-01-01')
          end
        end
        threads.each(&:join)
      end
    end
    ...
@headius
Copy link
Member

headius commented Mar 31, 2021

Can you get the full exception trace, preferably by passing -Xbacktrace.style=full to JRuby? I would like to see the exact internal line where this error is triggered.

@gerases
Copy link
Author

gerases commented Mar 31, 2021

Sure, please find below:

warning: thread "Ruby-0-Thread-6: /App/x2.rb:3" terminated with exception (report_on_exception is true):warning: thread "Ruby-0-Thread-41: /App/x2.rb:3" terminated with exception (report_on_exception is true):warning: thread "Ruby-0-Thread-24: /App/x2.rb:3" terminated with exception (report_on_exception is true):

java.lang.ClassCastException: org.jruby.RubyNil cannot be cast to org.jruby.RubyMatchData
        at org.jruby.RubyString.regexSplit(RubyString.java:4302)
        at org.jruby.RubyString.splitCommon(RubyString.java:4272)
        at org.jruby.RubyString.split(RubyString.java:4218)
        at org.jruby.RubyString$INVOKER$i$split.call(RubyString$INVOKER$i$split.gen)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:173)
        at App.x2.invokeOther0:split(/App/x2.rb:158)
        at App.x2.RUBY$block$\=\|App\|x2\,rb$1(/App/x2.rb:158)
        at App.x2.RUBY$block$run$7(/App/x2.rb:58)
        at App.x2.RUBY$block$process$16(/App/x2.rb:143)
        at org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:83)
        at org.jruby.runtime.Block.yieldSpecific(Block.java:162)
        at org.jruby.RubyIO$5.getline(RubyIO.java:2354)
        at org.jruby.RubyIO$5.getline(RubyIO.java:2348)
        at org.jruby.util.io.Getline.getlineCall(Getline.java:129)
        at org.jruby.util.io.Getline.getlineCall(Getline.java:39)
        at org.jruby.RubyIO.each(RubyIO.java:3329)
        at org.jruby.RubyIO$INVOKER$i$each.call(RubyIO$INVOKER$i$each.gen)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:152)
        at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:161)
        at App.x2.invokeOther82:each(/App/x2.rb:129)
        at App.x2.RUBY$block$process$14(/App/x2.rb:129)
        at org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
        at org.jruby.runtime.Block.yield(Block.java:184)
        at org.jruby.RubyIO.ensureYieldClose(RubyIO.java:1164)
        at org.jruby.RubyIO.open(RubyIO.java:1158)
        at org.jruby.RubyIO$INVOKER$s$0$0$open.call(RubyIO$INVOKER$s$0$0$open.gen)
        at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:204)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:183)
        at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:192)
        at App.x2.invokeOther88:open(/App/x2.rb:101)
        at App.x2.RUBY$method$process$13(/App/x2.rb:101)
        at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:93)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:152)
        at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:161)
        at App.x2.invokeOther16:process(/App/x2.rb:57)
        at App.x2.RUBY$block$run$6(/App/x2.rb:57)
        at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)
        at org.jruby.runtime.Block.call(Block.java:139)
        at org.jruby.RubyProc.call(RubyProc.java:318)
        at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
        at java.lang.Thread.run(Thread.java:748)
java.lang.ClassCastException: org.jruby.RubyNil cannot be cast to org.jruby.RubyMatchData
        at org.jruby.RubyString.regexSplit(RubyString.java:4302)
        at org.jruby.RubyString.splitCommon(RubyString.java:4272)
        at org.jruby.RubyString.split(RubyString.java:4218)
        at org.jruby.RubyString$INVOKER$i$split.call(RubyString$INVOKER$i$split.gen)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:173)
        at App.x2.invokeOther0:split(/App/x2.rb:158)
        at App.x2.RUBY$block$\=\|App\|x2\,rb$1(/App/x2.rb:158)
        at App.x2.RUBY$block$run$7(/App/x2.rb:58)
        at App.x2.RUBY$block$process$16(/App/x2.rb:143)
        at org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at org.jruby.runtime.IRBlockBody.yieldSpecific(IRBlockBody.java:83)
        at org.jruby.runtime.Block.yieldSpecific(Block.java:162)
        at org.jruby.RubyIO$5.getline(RubyIO.java:2354)
        at org.jruby.RubyIO$5.getline(RubyIO.java:2348)
        at org.jruby.util.io.Getline.getlineCall(Getline.java:129)
        at org.jruby.util.io.Getline.getlineCall(Getline.java:39)
        at org.jruby.RubyIO.each(RubyIO.java:3329)
        at org.jruby.RubyIO$INVOKER$i$each.call(RubyIO$INVOKER$i$each.gen)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:152)
        at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:161)
        at App.x2.invokeOther82:each(/App/x2.rb:129)
        at App.x2.RUBY$block$process$14(/App/x2.rb:129)
        at org.jruby.runtime.CompiledIRBlockBody.yieldDirect(CompiledIRBlockBody.java:148)
        at org.jruby.runtime.BlockBody.yield(BlockBody.java:106)
        at org.jruby.runtime.Block.yield(Block.java:184)
        at org.jruby.RubyIO.ensureYieldClose(RubyIO.java:1164)
        at org.jruby.RubyIO.open(RubyIO.java:1158)
        at org.jruby.RubyIO$INVOKER$s$0$0$open.call(RubyIO$INVOKER$s$0$0$open.gen)
        at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:204)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:183)
        at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:192)
        at App.x2.invokeOther88:open(/App/x2.rb:101)
        at App.x2.RUBY$method$process$13(/App/x2.rb:101)
        at org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:93)
        at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:152)
        at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:161)
        at App.x2.invokeOther16:process(/App/x2.rb:57)
        at App.x2.RUBY$block$run$6(/App/x2.rb:57)
        at org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:138)
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:58)
        at org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:52)
        at org.jruby.runtime.Block.call(Block.java:139)
        at org.jruby.RubyProc.call(RubyProc.java:318)
        at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)
        at java.lang.Thread.run(Thread.java:748)

@gerases
Copy link
Author

gerases commented Mar 31, 2021

If needed, I can attach the full code that creates the situation

@headius
Copy link
Member

headius commented Mar 31, 2021

I have confirmed that this is indeed the same issue as #4868. We cannot avoid the threading issues with $~ until CRuby decides it should be fixed, but I believe I can eliminate most of the concurrency issues here by avoiding the shared storage during split processes.

@gerases If you can provide a reproduction that would be great! I will need to be able to confirm my fix and write an appropriate test for it.

@gerases
Copy link
Author

gerases commented Mar 31, 2021

@headius, you mean you would like me to provide the source code? It's no problem, just want to make sure I understand you correctly.

@headius
Copy link
Member

headius commented Mar 31, 2021

Actually I think I have a small reproduction already:

str = 'a,b,c,d,e'
Thread.abort_on_exception = true
p = proc { str.split(/,/) }
10.times.map { Thread.new { loop { p.call } } }.each(&:join)

Which eventually yields:

java.lang.ClassCastException: class org.jruby.RubyNil cannot be cast to class org.jruby.RubyMatchData (org.jruby.RubyNil and org.jruby.RubyMatchData are in module org.jruby.dist of loader 'app')
	at org.jruby.dist/org.jruby.RubyString.regexSplit(RubyString.java:4302)
	at org.jruby.dist/org.jruby.RubyString.splitCommon(RubyString.java:4272)
	at org.jruby.dist/org.jruby.RubyString.split(RubyString.java:4218)
	at org.jruby.dist/org.jruby.RubyString$INVOKER$i$split.call(RubyString$INVOKER$i$split.gen)
	at org.jruby.dist/org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:172)
	at DashE.invokeOther0:split(-e:1)
	at DashE.RUBY$block$-e$0(-e:1)

This will be fixed by #6644. String#split will still be writing to a shared location for $~ but it never reads from it so it will at least not conflict with its own usage.

@gerases
Copy link
Author

gerases commented Mar 31, 2021

Cool, thank you very much!!!

headius added a commit to headius/jruby that referenced this issue Mar 31, 2021
See jruby#6640 for a failure that
occurred in JRuby due to using the context backref slot as an
"out" variable during regexp-based splitting.
@headius
Copy link
Member

headius commented Mar 31, 2021

@gerases Thank you for the bug report!

@headius headius linked a pull request Mar 31, 2021 that will close this issue
@headius headius added this to the JRuby 9.2.18.0 milestone Mar 31, 2021
@headius headius closed this as completed Mar 31, 2021
headius added a commit to headius/jruby that referenced this issue Apr 1, 2021
This changes all places we read the frame-local backref to use the
thread-local match instead. None of these places used the backref
for its content; they only used it to reuse the object, which is
now handled more correctly by the thread-local match and the
setBackRef method that marks matches as in-use.

This should eliminate all internal data races against the $~
variable, since no core methods depend upon its value. This does
not make $~ thread-safe but it pushes the issue to the edge, where
the user will have to decide to use it knowing it may be updated
across threads.

See jruby#4868 for the original issue that this partially fixes, and
see jruby#6640 for an issue caused by these internal data races.
headius added a commit to headius/jruby that referenced this issue Apr 1, 2021
This changes all places we read the frame-local backref to use the
thread-local match instead. None of these places used the backref
for its content; they only used it to reuse the object, which is
now handled more correctly by the thread-local match and the
setBackRef method that marks matches as in-use.

This should eliminate all internal data races against the $~
variable, since no core methods depend upon its value. This does
not make $~ thread-safe but it pushes the issue to the edge, where
the user will have to decide to use it knowing it may be updated
across threads.

See jruby#4868 for the original issue that this partially fixes, and
see jruby#6640 for an issue caused by these internal data races.
eregon pushed a commit to ruby/spec that referenced this issue Jun 2, 2021
See jruby/jruby#6640 for a failure that
occurred in JRuby due to using the context backref slot as an
"out" variable during regexp-based splitting.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants