Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
CompoundJarURLStreamHandler leaves many URL stream handles open before GC #4064
When initializing an application with a sufficiently large amount of JRuby ScriptingContainers (e.g., 32) and a sufficiently large maximum Java heap (e.g., 40Gb), we found that we were unable to get through the setup of all of the containers without encountering a "java.io.FileNotFoundException: (Too many open files)" exception when
We were able to work around the problem by bumping up
Ultimately, it appeared that the bulk of the file descriptors we saw allocated were coming from here. The
I put up a PR for closing each stream explicitly before it falls out of scope. The PR is #4063. I'm not totally sure if this is the right fix but it appears to be working well in our testing so far.
Provide at least:
jruby 1.7.25 (1.9.3p551) 2016-04-13 867cb81 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_45-b14 +jit [darwin-x86_64]
Version built from source, off the current HEAD of the jruby-1_7 branch, f83c032:
jruby 1.7.26-SNAPSHOT (1.9.3p551) 2016-08-10 f83c032 on Java HotSpot(TM) 64-Bit Server VM 1.8.0_45-b14 +jit [darwin-x86_64]
OSX El Capitan (10.11.6), CentOS 7, and others.
Steps to Reproduce
Using the reproducer at https://github.com/camlow325/jruby9k-benchmarks/tree/ae33418a140cb20cc32bc377b906c94a57f189f4, cd to “jar-file-desc-leak” and type:
This runs a Java application which creates
File descriptor usage remains relatively flat after each
This is the behavior I see with the fix in the linked PR. After each iteration, exactly the same number of file descriptors are open - 113.
File descriptor jumps to a peak of about 2200 across iterations. Open file descriptors may remain high even at the end of the run - 1200 or so - although this would drop back to around the 113 count from the expected case once Garbage Collection were triggered. Note that these counts are not deterministic given the relative unpredictability of when Garbage Collection may run.
This is the behavior I see on both JRuby 1.7.25 and 1.7.26-SNAPSHOT (built from the current head of the jruby-1_7 branch, f83c032.
@kares Do you know if there's a target release date for a JRuby 1.7.26 at this point? Is there anything we could do to help with getting a new release out? We're interested in using the fix for this issue and a few others that have landed recently in the jruby-1_7 branch as soon as may be feasible. Thanks again!