Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Can't require in two threads at once in JRuby 1.7.5.dev #993

Closed
bensummers opened this Issue · 17 comments

3 participants

@bensummers

If you call require in one thread, and while the code is being executed, call require in another thread, the second thread will hang.

This is changed behaviour from 1.7.4.

Test case: https://gist.github.com/bensummers/6484475

When running the test case:

$ jruby -v test.rb
jruby 1.7.4 (1.9.3p392) 2013-05-16 2390d3b on Java HotSpot(TM) Server VM 1.7.0_25-b15 [SunOS-x86]
Boo!
(exits normally)

$ jruby -v test.rb
jruby 1.7.5.dev (1.9.3p392) 2013-09-07 06d6243 on Java HotSpot(TM) Server VM 1.7.0_25-b15 [SunOS-x86]

(hangs, never returns)

@BanzaiMan
Owner

FYI: This test case hangs forever on 2.x as well.

If you put a Timeout around it, Timeout::Error is thrown without printing Boo!. JRuby master prints Boo!.

1.9.3 and 1.8.7 print Boo! and exits.

@headius
Owner

Hmm... this seems like it should be able to complete. The thread that requires 'b' from within 'a' should do the require and complete successfully before the second 'b' require happens.

@headius
Owner

Ok, it looks like this is RubyGems... in kernel_require before it goes to do require logic and potentially activate gems, it acquires a lock. This lock is held until require finishes, meaning that you can't do requires across threads. May be a RG 2.1 addition...I'm investigating.

@headius
Owner

Ok...so it looks like this is a "feature" of RubyGems 2.0 added in rubygems/rubygems@fcde0cd. The commit claims this is to "protect the require code" and admits that it will cause requires to serialize. So, it appears this is a by-design feature of RG 2.0?

I'm going to talk with @evanphx about the change.

@bensummers

In our app, we use require to load the bootstrap code for the main application and the test runner. While this does all the 'official' requires, other threads do things which trigger ActiveSupport's auto-loading.

Perhaps there's a better way of loading and running some code, which then waits for stuff to happen in other threads?

@headius
Owner

I think I have a possible patch for RG that moves the final require of code out of the protected zone, allowing requires to fire concurrently while still protecting RG internals.

@headius
Owner

I have filed rubygems/rubygems#637 to possibly move the actual file loading out of the locked section, which would fix this issue.

@headius
Owner

My patch for rubygems/rubygems#637 did not pass tests and so it did not make it into RG 2.1. We are still going to go forward with RG 2.1 in JRuby 1.7.5, however, so I have come up with a less invasive patch that appears to pass tests.

@bensummers can you try this patch against JRuby master and see if it solves your issues?

https://gist.github.com/headius/6516066

@headius
Owner

Updated patch to release monitor for all gem_original_require paths.

@headius
Owner

Ok, so the new patch has been merged into RubyGems for rubygems/rubygems#640 and will be in RG 2.1.2 release tomorrow. I think that will resolve this case.

@bensummers

@headius Shall I wait until you have RubyGems 2.1.2 before giving this a go?

@headius
Owner

@bensummers You could test it now by installing RubyGems master, which has the fix and which will become 2.1.2 later today. Run jruby setup.rb. You may need to revert a change the RubyGems setup does to bin/jgem (removes env and replaces it with full path to jruby, which doesn't work if jruby is a bash script).

Otherwise we'll have it installed on master whenever it is released by @drbrain today.

@BanzaiMan
Owner

@bensummers Please test c73c45e.

@bensummers

The build is failing at the "ant jar-complete" stage, with an error which looks a bit like it's gem related:

$ ant jar-complete
Buildfile: /export/home/ben/jruby-c73c45e4aa4fa0b796880136c08499aedc9a8d6a/build.xml

init:

prepare-bin-jruby:

jar-complete:

init:

jar-jruby:
     [exec] /export/home/ben/jruby-c73c45e4aa4fa0b796880136c08499aedc9a8d6a/core/target/generated-sources/annotated_classes.txt - not found. skip generator.

jar-jruby-complete:
     [copy] Copying 3 files to /export/home/ben/jruby-c73c45e4aa4fa0b796880136c08499aedc9a8d6a/target/jar-complete/META-INF/jruby.home
     [java] ERROR:  Could not find a valid gem '${complete.jar.gems}' (>= 0) in any repository

BUILD FAILED
/export/home/ben/jruby-c73c45e4aa4fa0b796880136c08499aedc9a8d6a/build.xml:291: The following error occurred while executing this line:
/export/home/ben/jruby-c73c45e4aa4fa0b796880136c08499aedc9a8d6a/build.xml:234: Java returned: 2

Total time: 1 minute 5 seconds
@BanzaiMan
Owner

We are moving away from ant for builds. Please try mvn -Pcomplete. You will find jruby-complete.jar in maven/jruby-complete/target/jruby-complete-1.7.5.dev.jar.

You can also find it in http://ci.jruby.org/snapshots/master/.

@bensummers

I was following the instructions here: https://github.com/jruby/jruby/wiki/DownloadAndBuildJRuby !

Using the snapshot was much easier, and my tests run to completion. So I believe this issue is resolved.

The tests didn't all pass though, and I think they suggest a Java integration issue. I will try and get a test case ASAP for you.

Thank you!

@BanzaiMan
Owner

@bensummers Ah, thanks for pointing to the outdated Wiki page.

I'm closing this one, but if you narrow down the problem with Java Integration, we'll take a look.

Thank you.

@BanzaiMan BanzaiMan closed this
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.