Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression with Scala 2.13: cannot compile GraalVM native-image #11634

Open
plokhotnyuk opened this issue Jul 15, 2019 · 11 comments

Comments

@plokhotnyuk
Copy link

@plokhotnyuk plokhotnyuk commented Jul 15, 2019

An example which successfully compiles to native binaries from Scala 2.11 & 2.12 uber-jars cannot be compiled from Scala 2.13 uber-jar due some internal changes in Scala collection implementation.

Error output from the native-image tool is:

Error: Unsupported features in 5 methods
Detailed message:
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: java.lang.Object[].clone(). To diagnose the issue you can use the --allow-incomplete-classpath option. The missing method is then reported at run time when it is accessed the first time.
Trace: 
	at parsing scala.collection.immutable.VectorPointer.gotoPosWritable1(Vector.scala:1243)
Call path from entry point to scala.collection.immutable.VectorPointer.gotoPosWritable1(int, int, int, Object[]): 
	at scala.collection.immutable.VectorPointer.gotoPosWritable1(Vector.scala:1242)
	at scala.collection.immutable.VectorPointer.gotoPosWritable1$(Vector.scala:1241)
	at scala.collection.immutable.VectorBuilder.addAll(Vector.scala:840)
	at scala.collection.immutable.Vector$.from(Vector.scala:840)
	at scala.collection.immutable.Vector$.from(Vector.scala:27)
	at scala.collection.IterableFactory.apply(Factory.scala:104)
	at scala.collection.IterableFactory.apply$(Factory.scala:104)
	at scala.collection.immutable.List$.apply(List.scala:618)
	at scala.collection.SeqFactory$Delegate.apply(Factory.scala:305)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01$.main(Example01.scala:18)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01.main(Example01.scala)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:147)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: com.oracle.graal.pointsto.constraints.UnresolvedElementException: Discovered unresolved method during parsing: java.lang.Object[][].clone(). To diagnose the issue you can use the --allow-incomplete-classpath option. The missing method is then reported at run time when it is accessed the first time.
Trace: 
	at parsing scala.collection.immutable.VectorPointer.stabilize(Vector.scala:1197)
Call path from entry point to scala.collection.immutable.VectorPointer.stabilize(int): 
	at scala.collection.immutable.VectorPointer.stabilize(Vector.scala:1163)
	at scala.collection.immutable.VectorPointer.stabilize$(Vector.scala:1163)
	at scala.collection.immutable.VectorBuilder.addAll(Vector.scala:840)
	at scala.collection.immutable.Vector$.from(Vector.scala:840)
	at scala.collection.immutable.Vector$.from(Vector.scala:27)
	at scala.collection.IterableFactory.apply(Factory.scala:104)
	at scala.collection.IterableFactory.apply$(Factory.scala:104)
	at scala.collection.immutable.List$.apply(List.scala:618)
	at scala.collection.SeqFactory$Delegate.apply(Factory.scala:305)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01$.main(Example01.scala:18)
	at com.github.plokhotnyuk.jsoniter_scala.examples.Example01.main(Example01.scala)
	at com.oracle.svm.core.JavaMainWrapper.run(JavaMainWrapper.java:147)
	at com.oracle.svm.core.code.IsolateEnterStub.JavaMainWrapper_run_5087f5482cc9a6abc971913ece43acb471d2631b(generated:0)
Error: com.oracle.svm.hosted.substitute.DeletedElementException: Unsupported type java.lang.invoke.MemberName is reachable: All methods from java.lang.invoke should have been replaced during image building.
To diagnose the issue, you can add the option --report-unsupported-elements-at-runtime. The unsupported element is then reported at run time when it is accessed the first time.
Trace: 
	at parsing java.lang.invoke.MethodHandles$Lookup.findVirtual(MethodHandles.java:861)
Call path from entry point to java.lang.invoke.MethodHandles$Lookup.findVirtual(Class, String, MethodType): 
	no path found from entry point to target method

Error: type is not available in this platform: com.oracle.svm.hosted.ClassValueFeature
Trace: 	object java.lang.Class[]
	object java.lang.invoke.MethodType
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
	object java.util.concurrent.ConcurrentHashMap$Node
	object java.util.concurrent.ConcurrentHashMap$Node[]
	object java.util.concurrent.ConcurrentHashMap
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet
	method java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean)
Call path from entry point to java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean): 
	no path found from entry point to target method

Error: type is not available in this platform: com.oracle.svm.hosted.substitute.ComputedValueField
Trace: 	object java.lang.invoke.MethodType
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet$WeakEntry
	object java.util.concurrent.ConcurrentHashMap$Node
	object java.util.concurrent.ConcurrentHashMap$Node[]
	object java.util.concurrent.ConcurrentHashMap
	object java.lang.invoke.MethodType$ConcurrentWeakInternSet
	method java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean)
Call path from entry point to java.lang.invoke.MethodType.makeImpl(Class, Class[], boolean): 
	no path found from entry point to target method

Steps to reproduce:

  1. Install or make sure all those things are installed: Sbt, GraalVM CE/EE and its native-image tool
  2. Clone the following repo: https://github.com/plokhotnyuk/jsoniter-scala
  3. Checkout the switch-examples-to-scala-2.13 branch
  4. Run the following commands:
cd jsoniter-scala-examples
sbt clean +assembly
/usr/lib/jvm/graalvm-ce-19/bin/native-image --no-server --no-fallback -H:UnsafeAutomaticSubstitutionsLogLevel=3 -jar target/scala-2.13/jsoniter-scala-examples-assembly-0.1.0-SNAPSHOT.jar
@SethTisue

This comment has been minimized.

Copy link
Member

@SethTisue SethTisue commented Jul 15, 2019

is this a bug in Scala?

@plokhotnyuk

This comment has been minimized.

Copy link
Author

@plokhotnyuk plokhotnyuk commented Jul 15, 2019

IMHO, yes.

Some of these errors appear due to releaseFence calls, as reporred here. Others also could be prevented in case of opening some kind of community build for tools and libs which going to support AOT compilation mode, starting from scalac and core libraries.

@SethTisue

This comment has been minimized.

Copy link
Member

@SethTisue SethTisue commented Jul 15, 2019

are you saying that in order for this to work, we can't use releaseFence at all?

@plokhotnyuk

This comment has been minimized.

Copy link
Author

@plokhotnyuk plokhotnyuk commented Jul 15, 2019

There are implementation options to make Unsafe usage recognizable for the Graal compiler.

And for edge cases some cooperation with the GraalVM team will be required for sure.

I have managed to fix compilation error for releaseFence calls by the following commit: plokhotnyuk/jsoniter-scala@e089f06

Others calls to java.lang.Object[].clone() and java.lang.Object[][].clone() are still not substituted, and if they are not called in runtime then possible W/A is to use the --allow-incomplete-classpath option of the GraalVM native-image compiler.

@SethTisue

This comment has been minimized.

Copy link
Member

@SethTisue SethTisue commented Oct 2, 2019

Tentatively milestoned for 2.13.2, since it should certainly at least be assessed further.

@adriaanm

This comment has been minimized.

Copy link
Member

@adriaanm adriaanm commented Oct 3, 2019

Great that you managed to override the call to releaseFence. This (allowing other platforms to customize low level stuff) is why we provide that hook through scala.runtime.Statics. Maybe we can do something similar for the array clone methods?

@sjrd

This comment has been minimized.

Copy link
Member

@sjrd sjrd commented Oct 3, 2019

What's the problem with Array.clone()? They're not low-level at all; they're straightforward reflection-free user code:
https://github.com/scala/scala/blob/4ec48ff29755fc01fdde880dbb4548085420c601/src/library/scala/runtime/ScalaRunTime.scala#L94-L105

@adriaanm

This comment has been minimized.

Copy link
Member

@adriaanm adriaanm commented Oct 3, 2019

I meant java.lang.Object[].clone()

@adriaanm

This comment has been minimized.

Copy link
Member

@adriaanm adriaanm commented Oct 3, 2019

Anyway, that still seems a little too common for graal not to support natively? Quick google led to a similar, resolved issue: oracle/graal#877

@plokhotnyuk

This comment has been minimized.

Copy link
Author

@plokhotnyuk plokhotnyuk commented Oct 3, 2019

@adriaanm checked with GraalVM CE 19.2.0.1 and still got the same error...

now I'm waiting for the next release: 19.2.1 or 19.3.0 to check again...

@adriaanm

This comment has been minimized.

Copy link
Member

@adriaanm adriaanm commented Oct 3, 2019

Thanks for checking. Looks like the bug I linked to was reported fixed on the Java 11 version, which would be the upcoming 19.3, IUC.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.