/tmp/jrubyXXX.jar files are left in place by classloader if process is SIGKILLed #3352

Closed
rtyler opened this Issue Sep 25, 2015 · 7 comments

Comments

Projects
None yet
3 participants
@rtyler

rtyler commented Sep 25, 2015

When JRuby is executing via jruby-complete or another embedded type use-case, and it loads jars of some form or another a number of /tmp/jruby*.jar files are created.

Under "normal" conditions these files stick around and are removed by something when the process is sent SIGTERM or SIGINT.

A SIGKILL however will result in the files littering the /tmp directory indefinitely.

Reproduction Steps

  • java -cp ./jruby-complete-9.0.1.0.jar org.jruby.Main -e "puts 'hello'; require 'openssl'; puts 'loaded'; sleep 60"
  • ls /tmp | grep -i jar
➜  ~  ls /tmp | grep -i jar
jruby2147863957306982519jopenssl.jar
jruby3032390121297627542bcpkix-jdk15on-1.50.jar
jruby6589611494165220974bcprov-jdk15on-1.50.jar
➜  ~  
  • kill process with SIGKILL

Suggested solutions

It's not clear to me whether these files are needed after the classes are loaded into the runtime or not. If they are not needed, we could employ a similar fix to #1237 which was fixed with jnr/jffi#13

If they are needed for the duration of the runtime, then it would at least be useful to write the files with the PID of the Java process or something similar so a user can safely remove ones that are stale or no longer needed

@rtyler

This comment has been minimized.

Show comment
Hide comment
@rtyler

rtyler Sep 25, 2015

Thinking a bit outside the box here, we could use ShrinkWrap and their ShrinkWrapClassLoader which would obviate the need for JRubyClassLoader to copy files to /tmp

rtyler commented Sep 25, 2015

Thinking a bit outside the box here, we could use ShrinkWrap and their ShrinkWrapClassLoader which would obviate the need for JRubyClassLoader to copy files to /tmp

@rtyler rtyler assigned rtyler and unassigned mkristian Sep 25, 2015

@mkristian

This comment has been minimized.

Show comment
Hide comment
@mkristian

mkristian Sep 27, 2015

Member

@rtyler JRubyClassLoader is mutable as it can add jars and can defineClass anytime. the ShrinkWrapClassLoader has no such features and is not usable as such.

Member

mkristian commented Sep 27, 2015

@rtyler JRubyClassLoader is mutable as it can add jars and can defineClass anytime. the ShrinkWrapClassLoader has no such features and is not usable as such.

@mkristian

This comment has been minimized.

Show comment
Hide comment
@mkristian

mkristian Sep 27, 2015

Member

the files need to stay around as any resource loading can happen any time and the classloader does not cache the whole content in memory.

using the PID to collect the jars is possible but is NO guarantee since any forking of "ruby" will reuse those jars for setting up its "forked" jruby classpath. well, it would help for those apps which do not fork.

Member

mkristian commented Sep 27, 2015

the files need to stay around as any resource loading can happen any time and the classloader does not cache the whole content in memory.

using the PID to collect the jars is possible but is NO guarantee since any forking of "ruby" will reuse those jars for setting up its "forked" jruby classpath. well, it would help for those apps which do not fork.

@rtyler

This comment has been minimized.

Show comment
Hide comment
@rtyler

rtyler Sep 27, 2015

@mkristian Yes, I agree with the ShrinkWrapClassLoader no longer being viable. In my initial experiments it's just been too simplistic.

ShrinkWrap in general however, I believe will be a viable way to replace this code, most of what I'm struggling with around it is not the code so much as learning how these class fits into the rest of the LoadService.

I think ShrinkWrap is the right way forward here, and it's not that difficult to use ShrinkWrap, just a tough task for me as a core newbie :)

rtyler commented Sep 27, 2015

@mkristian Yes, I agree with the ShrinkWrapClassLoader no longer being viable. In my initial experiments it's just been too simplistic.

ShrinkWrap in general however, I believe will be a viable way to replace this code, most of what I'm struggling with around it is not the code so much as learning how these class fits into the rest of the LoadService.

I think ShrinkWrap is the right way forward here, and it's not that difficult to use ShrinkWrap, just a tough task for me as a core newbie :)

@mkristian

This comment has been minimized.

Show comment
Hide comment
@mkristian

mkristian Sep 28, 2015

Member

@rtyler the LoadService is only part of the picture. I added some javadocs to JRubyClassLoader https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/util/JRubyClassLoader.java#L45

the JRubyClassLoader.addURL needs to work with any possible custom url protocol. since the passed in URL might not work like this: {code}new URL(url.toString()).openStream(){code} the copy to /tmp is there. some classloaders produce URLs with "special" StreamHandler which do not work after the round trip over the String. having the URLClassLoader deal with only "file:" URLs has proven to work with all kind of custom classloaders coming with j2ee or osgi.

further the URLClassLoader bits are also used to find directory entries in case there are no '.jrubydir' files on these added jars. not a big usecase IMO but was a legacy requirement to keep.

the LoadService itself is not the right place and org.jruby.runtime.load.LibrarySearcher is the better or any code which uses JRubyClassLoader.addURL

Member

mkristian commented Sep 28, 2015

@rtyler the LoadService is only part of the picture. I added some javadocs to JRubyClassLoader https://github.com/jruby/jruby/blob/master/core/src/main/java/org/jruby/util/JRubyClassLoader.java#L45

the JRubyClassLoader.addURL needs to work with any possible custom url protocol. since the passed in URL might not work like this: {code}new URL(url.toString()).openStream(){code} the copy to /tmp is there. some classloaders produce URLs with "special" StreamHandler which do not work after the round trip over the String. having the URLClassLoader deal with only "file:" URLs has proven to work with all kind of custom classloaders coming with j2ee or osgi.

further the URLClassLoader bits are also used to find directory entries in case there are no '.jrubydir' files on these added jars. not a big usecase IMO but was a legacy requirement to keep.

the LoadService itself is not the right place and org.jruby.runtime.load.LibrarySearcher is the better or any code which uses JRubyClassLoader.addURL

@mkristian

This comment has been minimized.

Show comment
Hide comment
@mkristian

mkristian Sep 28, 2015

Member

let's go the PID idea first and give more time to the ShrinkWrapClassloader idea as it is a bigger change and hard to tell where problems pop up

Member

mkristian commented Sep 28, 2015

let's go the PID idea first and give more time to the ShrinkWrapClassloader idea as it is a bigger change and hard to tell where problems pop up

@mkristian

This comment has been minimized.

Show comment
Hide comment
@mkristian

mkristian Sep 29, 2015

Member

improved with 28f6507

closing and continue the ShrinkWrap idea here: #3361

Member

mkristian commented Sep 29, 2015

improved with 28f6507

closing and continue the ShrinkWrap idea here: #3361

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment