New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use of JRUBY_OPTS="-Xreify.classes=true" causes Java::JavaLang::StackOverflowError: #5266

Open
voyager131 opened this Issue Jul 29, 2018 · 1 comment

Comments

Projects
None yet
2 participants
@voyager131

voyager131 commented Jul 29, 2018

Please see issue 5249 for a refresher on what I'm doing. Here's my current environment:

user@host:~/keyfile_data$ java -version
openjdk version "1.8.0_171"
OpenJDK Runtime Environment (build 1.8.0_171-8u171-b11-0ubuntu0.18.04.1-b11)
OpenJDK 64-Bit Server VM (build 25.171-b11, mixed mode)
user@host:~/keyfile_data$ jruby --version
jruby 9.2.0.0 (2.5.0) 2018-05-24 81156a8 OpenJDK 64-Bit Server VM 25.171-b11 on 1.8.0_171-8u171-b11-0ubuntu0.18.04.1-b11 +jit [linux-x86_64]

Under normal working conditions...

JRUBY_OPTS="-Xmanagement.enabled=true -Xcompile.mode=FORCE"

I'm trying to add reify.classes to the mix as follows without success...

JRUBY_OPTS="-Xmanagement.enabled=true -Xcompile.mode=FORCE -Xreify.classes=true"

The Problem
The previous little conversion binary discussed in issue 5249 is part of a much bigger piece of software I'm writing in ruby that processes hundreds of thousands of files. Jruby has been a lifesaver because it supports true parallelism and is not subject to MRI ruby's global interpreter lock (GIL). Using jruby and Celluloid, (now that I'm on Java 8), I've been able to take code that used to take 8+ hours to process a test set of ~800,000 files down to just over an hour. As I've been benchmarking the code (using VisualVM), I have reason to believe it could be even faster if I could identify the memory inefficiencies that appear to be present. However, I'm fit to be tied to correlate what I see in VisualVM or MemoryAnalyzer (MAT), to my ruby code. From my research, it appears that I need to be able to use -Xreify.classes=true in my JRUBY_OPTS, which according to @headius, will "stand up a JVM class for every user-created Ruby class...", and give us among other benefits "...a more useful profile." (Browsing Memory the JRuby Way).

The reference is old and I'm new to all of this, but notwithstanding several hours of googling, I can't seem to find the solution. I also found this, and wonder if it's not somehow related to my problem.

Just to be clear, without reify.classes=true, my code works just fine though making sense of the data in VisualVM or MemoryAnalyzer is nearly impossible. Can someone help? Here's part of the exception dump that happens when I add reify.classes=true to JRUBY_OPTS:

E, [2018-07-29T11:04:41.728003 #14329] ERROR -- : Actor crashed!
Java::JavaLang::StackOverflowError: 
	sun.reflect.generics.parser.SignatureParser.parseClassTypeSignature(SignatureParser.java:307)
	sun.reflect.generics.parser.SignatureParser.parseFieldTypeSignature(SignatureParser.java:289)
	sun.reflect.generics.parser.SignatureParser.parseFieldTypeSignature(SignatureParser.java:283)
	sun.reflect.generics.parser.SignatureParser.parseTypeSignature(SignatureParser.java:485)
	sun.reflect.generics.parser.SignatureParser.parseTypeSig(SignatureParser.java:188)
	sun.reflect.annotation.AnnotationParser.parseSig(AnnotationParser.java:436)
	sun.reflect.annotation.AnnotationParser.parseAnnotation2(AnnotationParser.java:241)
	sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:120)
	sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:72)
	java.lang.reflect.Executable.declaredAnnotations(Executable.java:599)
	java.lang.reflect.Executable.declaredAnnotations(Executable.java:597)
	java.lang.reflect.Executable.getAnnotation(Executable.java:570)
	java.lang.reflect.Method.getAnnotation(Method.java:622)
	org.jruby.RubyModule$MethodClumper.clump(RubyModule.java:1127)
	org.jruby.anno.TypePopulator$ReflectiveTypePopulator.<init>(TypePopulator.java:91)
	org.jruby.RubyModule.loadPopulatorFor(RubyModule.java:1202)
	org.jruby.Ruby$8.computeValue(Ruby.java:5298)
	org.jruby.Ruby$8.computeValue(Ruby.java:5295)
	java.lang.ClassValue.getFromHashMap(ClassValue.java:227)
	java.lang.ClassValue.getFromBackup(ClassValue.java:209)
	java.lang.ClassValue.get(ClassValue.java:115)
	org.jruby.RubyModule.defineAnnotatedMethodsIndividually(RubyModule.java:1206)
	org.jruby.RubyModule.defineAnnotatedMethods(RubyModule.java:1113)
	org.jruby.javasupport.ext.JavaLang$Throwable.define(JavaLang.java:209)
	org.jruby.javasupport.ext.JavaLang.lambda$define$2(JavaLang.java:64)
	org.jruby.javasupport.ext.JavaExtensions.define(JavaExtensions.java:31)
	org.jruby.javasupport.JavaSupportImpl$3.computeValue(JavaSupportImpl.java:129)
	org.jruby.javasupport.JavaSupportImpl$3.computeValue(JavaSupportImpl.java:120)
	org.jruby.util.collections.MapBasedClassValue.get(MapBasedClassValue.java:25)
	org.jruby.javasupport.JavaSupportImpl.getProxyClassFromCache(JavaSupportImpl.java:203)
	org.jruby.javasupport.Java.getProxyClass(Java.java:424)
	org.jruby.javasupport.Java.createProxyClassForClass(Java.java:446)
	org.jruby.javasupport.JavaSupportImpl$3.computeValue(JavaSupportImpl.java:128)
	org.jruby.javasupport.JavaSupportImpl$3.computeValue(JavaSupportImpl.java:120)
	org.jruby.util.collections.MapBasedClassValue.get(MapBasedClassValue.java:25)
	org.jruby.javasupport.JavaSupportImpl.getProxyClassFromCache(JavaSupportImpl.java:203)
	org.jruby.javasupport.Java.getProxyClass(Java.java:424)
	org.jruby.javasupport.Java.createProxyClassForClass(Java.java:446)
	org.jruby.javasupport.JavaSupportImpl$3.computeValue(JavaSupportImpl.java:128)
	org.jruby.javasupport.JavaSupportImpl$3.computeValue(JavaSupportImpl.java:120)
	org.jruby.util.collections.MapBasedClassValue.get(MapBasedClassValue.java:25)
	org.jruby.javasupport.JavaSupportImpl.getProxyClassFromCache(JavaSupportImpl.java:203)
	org.jruby.javasupport.Java.getProxyClass(Java.java:424)
	org.jruby.javasupport.Java.createProxyClassForClass(Java.java:446)
	org.jruby.javasupport.JavaSupportImpl$3.computeValue(JavaSupportImpl.java:128)
	org.jruby.javasupport.JavaSupportImpl$3.computeValue(JavaSupportImpl.java:120)
	org.jruby.util.collections.MapBasedClassValue.get(MapBasedClassValue.java:25)
	org.jruby.javasupport.JavaSupportImpl.getProxyClassFromCache(JavaSupportImpl.java:203)
	org.jruby.javasupport.Java.getProxyClass(Java.java:424)
	org.jruby.javasupport.Java.getInstance(Java.java:351)
	org.jruby.javasupport.Java.getInstance(Java.java:333)
	org.jruby.javasupport.JavaUtil.convertJavaToUsableRubyObject(JavaUtil.java:214)
	org.jruby.runtime.Helpers.wrapJavaException(Helpers.java:790)
	org.jruby.ir.runtime.IRRuntimeHelpers.wrapJavaException(IRRuntimeHelpers.java:390)
	org.jruby.ir.runtime.IRRuntimeHelpers.isJavaExceptionHandled(IRRuntimeHelpers.java:375)
	org.jruby.ir.runtime.IRRuntimeHelpers.isJavaExceptionHandled(IRRuntimeHelpers.java:361)
	org.jruby.ir.runtime.IRRuntimeHelpers.isExceptionHandled(IRRuntimeHelpers.java:435)
	home.user.$_dot_rbenv.versions.jruby_minus_9_dot_2_dot_0_dot_0.lib.ruby.gems.shared.gems.celluloid_minus_pool_minus_0_dot_20_dot_5.lib.celluloid.supervision.container.pool_dot_rb.RUBY$method$method$41(/home/user/.rbenv/versions/jruby-9.2.0.0/lib/ruby/gems/shared/gems/celluloid-pool-0.20.5/lib/celluloid/supervision/container/pool.rb:203)
	org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:166)
	org.jruby.RubyClass.finvoke(RubyClass.java:911)
	org.jruby.runtime.Helpers.invoke(Helpers.java:356)
	org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:386)
	rubyobj.Celluloid.Supervision.Container.Pool.method(Unknown Source)
	org.jruby.RubyKernel.method(RubyKernel.java:2150)
	org.jruby.RubyKernel$INVOKER$s$1$0$method.call(RubyKernel$INVOKER$s$1$0$method.gen)
	org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:815)
	org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1081)
	home.user.$_dot_rbenv.versions.jruby_minus_9_dot_2_dot_0_dot_0.lib.ruby.gems.shared.gems.celluloid_minus_pool_minus_0_dot_20_dot_5.lib.celluloid.supervision.container.pool_dot_rb.invokeSuper218:method(/home/user/.rbenv/versions/jruby-9.2.0.0/lib/ruby/gems/shared/gems/celluloid-pool-0.20.5/lib/celluloid/supervision/container/pool.rb:203)
	home.user.$_dot_rbenv.versions.jruby_minus_9_dot_2_dot_0_dot_0.lib.ruby.gems.shared.gems.celluloid_minus_pool_minus_0_dot_20_dot_5.lib.celluloid.supervision.container.pool_dot_rb.RUBY$method$method$41(/home/user/.rbenv/versions/jruby-9.2.0.0/lib/ruby/gems/shared/gems/celluloid-pool-0.20.5/lib/celluloid/supervision/container/pool.rb:203)
	org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:166)
	org.jruby.RubyClass.finvoke(RubyClass.java:911)
	org.jruby.runtime.Helpers.invoke(Helpers.java:356)
	org.jruby.RubyBasicObject.callMethod(RubyBasicObject.java:386)
	rubyobj.Celluloid.Supervision.Container.Pool.method(Unknown Source)
	org.jruby.RubyKernel.method(RubyKernel.java:2150)
	org.jruby.RubyKernel$INVOKER$s$1$0$method.call(RubyKernel$INVOKER$s$1$0$method.gen)
	org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:815)
	org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1081)
	home.user.$_dot_rbenv.versions.jruby_minus_9_dot_2_dot_0_dot_0.lib.ruby.gems.shared.gems.celluloid_minus_pool_minus_0_dot_20_dot_5.lib.celluloid.supervision.container.pool_dot_rb.invokeSuper218:method(/home/user/.rbenv/versions/jruby-9.2.0.0/lib/ruby/gems/shared/gems/celluloid-pool-0.20.5/lib/celluloid/supervision/container/pool.rb:203)
	home.user.$_dot_rbenv.versions.jruby_minus_9_dot_2_dot_0_dot_0.lib.ruby.gems.shared.gems.celluloid_minus_pool_minus_0_dot_20_dot_5.lib.celluloid.supervision.container.pool_dot_rb.RUBY$method$method$41(/home/user/.rbenv/versions/jruby-9.2.0.0/lib/ruby/gems/shared/gems/celluloid-pool-0.20.5/lib/celluloid/supervision/container/pool.rb:203)
	org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:166)

It keeps going but at this point we appear to be in a loop of some sort.

@headius

This comment has been minimized.

Show comment
Hide comment
@headius

headius Jul 29, 2018

Member

This is unfortunately a known issue combining reified classes with reified instance variables. It should work if you also pass -Xreify.variables=false I believe.

I have reworked variable reification for 9.2.1 and it should be easier to get reified classes (at least so they have proper names) to work now too.

Member

headius commented Jul 29, 2018

This is unfortunately a known issue combining reified classes with reified instance variables. It should work if you also pass -Xreify.variables=false I believe.

I have reworked variable reification for 9.2.1 and it should be easier to get reified classes (at least so they have proper names) to work now too.

@headius headius added this to the JRuby 9.2.1.0 milestone Jul 29, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment