Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with HTTPS or Subversion.

Download ZIP

Loading…

Direct Memory Buffer Leak in File IO #863

Closed
stewi2 opened this Issue · 3 comments

3 participants

@stewi2

Apparently the JRuby File class allocates NIO direct memory buffers, but doesn't release them. I wrote the following test program (on jruby-1.7.4 on Java 1.7 on OSX 10.8.4):

i=0
loop do
  i=i+1
  File.open('/dev/null','r') do |infile|
  end
  puts i
  sleep 0.1
end

When I run this with a restricted MaxDirectMemorySize using:

jruby  -J-XX:MaxDirectMemorySize=200K -w leak.rb

it crashes very quickly:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
Error: Your application used more memory than the safety cap of 500M.
Specify -J-Xmx####m to increase it (#### = cap size in MB).
Exception trace follows:
java.lang.OutOfMemoryError: Direct buffer memory
    at java.nio.Bits.reserveMemory(Bits.java:658)
    at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:123)
    at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:306)
    at org.jruby.util.io.ChannelDescriptor.<init>(ChannelDescriptor.java:663)
    at org.jruby.util.io.ChannelDescriptor.<init>(ChannelDescriptor.java:217)
    at org.jruby.util.io.ChannelDescriptor.open(ChannelDescriptor.java:797)
    at org.jruby.RubyFile.sysopen(RubyFile.java:1242)
    at org.jruby.RubyFile.sysopenInternal(RubyFile.java:1199)
    at org.jruby.RubyFile.openFile19(RubyFile.java:1150)
    at org.jruby.RubyFile.initialize19(RubyFile.java:334)
    at org.jruby.RubyFile$INVOKER$i$0$2$initialize19.call(RubyFile$INVOKER$i$0$2$initialize19.gen)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
    at org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:85)
    at org.jruby.RubyClass.newInstance(RubyClass.java:876)
    at org.jruby.RubyIO.open(RubyIO.java:1144)
    at org.jruby.RubyIO$INVOKER$s$0$0$open.call(RubyIO$INVOKER$s$0$0$open.gen)
    at org.jruby.internal.runtime.methods.DynamicMethod.call(DynamicMethod.java:217)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:211)
    at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:222)
    at leak.block_0$RUBY$__file__(leak.rb:6)
    at leak$block_0$RUBY$__file__.call(leak$block_0$RUBY$__file__)
    at org.jruby.runtime.CompiledBlock19.yieldSpecificInternal(CompiledBlock19.java:117)
    at org.jruby.runtime.CompiledBlock19.yieldSpecific(CompiledBlock19.java:92)
    at org.jruby.runtime.Block.yieldSpecific(Block.java:111)
    at org.jruby.RubyKernel.loop(RubyKernel.java:1489)
    at org.jruby.RubyKernel$INVOKER$s$0$0$loop.call(RubyKernel$INVOKER$s$0$0$loop.gen)
    at org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:316)
    at org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:145)
    at org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:154)
    at leak.__file__(leak.rb:4)
    at leak.load(leak.rb)
    at org.jruby.Ruby.runScript(Ruby.java:807)
    at org.jruby.Ruby.runScript(Ruby.java:800)
    at org.jruby.Ruby.runNormally(Ruby.java:669)
    at org.jruby.Ruby.runFromMain(Ruby.java:518)
    at org.jruby.Main.doRunFromMain(Main.java:390)
    at org.jruby.Main.internalRun(Main.java:279)
    at org.jruby.Main.run(Main.java:221)
    at org.jruby.Main.main(Main.java:201)

I tried the same thing by simply opening and closing a FileInputStream, and the memory usage stays constant. I used https://blogs.oracle.com/nbprofiler/entry/new_memory_buffer_plugin_for to monitor buffer usage.

@atambo
Collaborator

I ran your script against 1.7.4 and reproduced your OutOfMemoryError. But when I ran it against master it didn't run out of memory so looks like it is already fixed and will be in 1.7.5.

@atambo atambo closed this
@headius
Owner

I also ran this on Java 6 and Java 7. Java 6 stabilized very early at around 112MB. I let it run for about 8000 iterations (after dropping the sleep down to 0.01s). Java 7 stabilized higher, around 171MB, after running up to 109k iterations. I'm convinced that this is fixed as well.

Free free to confirm with master and reopen if there are any further issues. Thanks for the report!

@stewi2

Thanks guys for the quick response. I also verified that this does appear to be fixed in master.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Something went wrong with that request. Please try again.