Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

JDK 11 - Class sun.reflect.CallerSensitive not found - continuing with a stub. #11679

Closed
Sciss opened this issue Aug 13, 2019 · 11 comments · Fixed by sbt/zinc#701
Closed

JDK 11 - Class sun.reflect.CallerSensitive not found - continuing with a stub. #11679

Sciss opened this issue Aug 13, 2019 · 11 comments · Fixed by sbt/zinc#701
Labels

Comments

@Sciss
Copy link

Sciss commented Aug 13, 2019

I am building a project with "-release", "8", compiling with sbt under OpenJDK 11. For both cross Scala versions 2.12.9 and 2.13.0, this fails at several places with errors like

[error] /home/hhrutz/Documents/devel/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/SynthGraphObj.scala:166:27: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]           val companion = Class.forName(s"$className").getField("MODULE$").get(null)
[error]                           ^

or

[error] /home/hhrutz/Documents/devel/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/impl/MemoryClassLoader.scala:20:39: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error] final class MemoryClassLoader extends ClassLoader {
[error]                                       ^

Unfortunately it's not easy to isolate the code that breaks (I tried simple reductions but they don't crash the compiler). It's on a project that runs on multiple snapshot dependencies, so the heavy way to reproduce is

java -version
openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment (build 11.0.4+11-post-Debian-1deb10u1)
OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Debian-1deb10u1, mixed mode, sharing)

(i.e. use JDK 11)

git clone https://github.com/Sciss/Lucre.git
cd Lucre
git checkout -b debug 53c779f0edfbf6bf0fc8f276d946290c145d85e3
sbt update
sbt publishLocal
cd ..
git clone https://github.com/Sciss/LucreSwing.git
cd LucreSwing
git checkout -b debug e6b50892553e765a3e840ea68c4bf5e5e0879fd9
sbt update
sbt publishLocal
cd ..
git clone https://github.com/Sciss/SoundProcesses.git
cd SoundProcesses
git checkout -b debug f25d534f5f74b80c3cc52abbec5fcbbed3958e9d
sbt update
sbt compile

This should be reproducible and yield

[error] /data/temp/bla/SoundProcesses/core/src/main/scala/de/sciss/synth/proc/SynthGraphObj.scala:166:27: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]           val companion = Class.forName(className).getField("MODULE$").get(null)
[error]                           ^

Perhaps related to #10817 . The only place that CallerSensitive appears in seems to be https://github.com/scala/scala/blob/3027652cab884c88f5cc20f710e351a0730c5ba2/src/compiler/scala/tools/nsc/backend/jvm/opt/BytecodeUtils.scala#L132

@SethTisue
Copy link
Member

@lrytz this one might interest you

@SethTisue SethTisue added this to the Backlog milestone Aug 21, 2019
@lrytz
Copy link
Member

lrytz commented Aug 23, 2019

git clone https://github.com/Sciss/Lucre.git
cd Lucre
git checkout -b debug 53c779f0edfbf6bf0fc8f276d946290c145d85e3
sbt update

failed already for me

[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	::          UNRESOLVED DEPENDENCIES         ::
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn] 	:: com.sleepycat#je;6.4.25: not found
[warn] 	::::::::::::::::::::::::::::::::::::::::::::::
[warn]
[warn] 	Note: Unresolved dependencies path:
[warn] 		com.sleepycat:je:6.4.25 (/Users/luc/Downloads/tmp/Lucre/build.sbt#L171)
[warn] 		  +- de.sciss:lucre-bdb6_2.12:3.14.0-SNAPSHOT
[error] sbt.librarymanagement.ResolveException: unresolved dependency: com.sleepycat#je;7.4.5: not found

I don't think it's related to the code in the backend that you refer to, as we're not loading any symbols at this point.

It's probably some other entity that carries the @CallerSensitive annotation causing the symbol to be loaded.

It's quite certainly related to the use of -release. Since CallerSensitive is part of sun.reflect, does it actually exists in the ct.sym file? cc @retronym.

@Sciss
Copy link
Author

Sciss commented Aug 23, 2019

failed already for me

Oh great, Oracle is doing its best to tear down all their Java related resources (https://download.oracle.com/maven - 404). It will take me a moment to see where that repository went.

@Sciss
Copy link
Author

Sciss commented Aug 23, 2019

Hmmm, no, download works fine for me (and Travis CI also builds without complaining); perhaps you are behind a strange firewall?

...
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/6.4.25/je-6.4.25.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;6.4.25!je.jar (4266ms)
[info] Done updating.
[info] downloading https://repo1.maven.org/maven2/de/sciss/fileutil_2.12/1.1.3/fileutil_2.12-1.1.3.jar ...
[info] downloading https://repo1.maven.org/maven2/de/sciss/model_2.12/0.3.4/model_2.12-0.3.4.jar ...
[info] downloading https://repo1.maven.org/maven2/de/sciss/span_2.12/1.4.2/span_2.12-1.4.2.jar ...
[info] 	[SUCCESSFUL ] de.sciss#fileutil_2.12;1.1.3!fileutil_2.12.jar (286ms)
[info] 	[SUCCESSFUL ] de.sciss#model_2.12;0.3.4!model_2.12.jar (286ms)
[info] 	[SUCCESSFUL ] de.sciss#span_2.12;1.4.2!span_2.12.jar (463ms)
[info] Done updating.
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/5.0.104/je-5.0.104.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;5.0.104!je.jar (3239ms)
[info] Done updating.
[info] downloading http://download.oracle.com/maven/com/sleepycat/je/7.4.5/je-7.4.5.jar ...
[info] 	[SUCCESSFUL ] com.sleepycat#je;7.4.5!je.jar (4582ms)
[info] Done updating.
[info] downloading https://repo1.maven.org/maven2/de/sciss/fingertree_2.12/1.5.4/fingertree_2.12-1.5.4.jar ...
[info] 	[SUCCESSFUL ] de.sciss#fingertree_2.12;1.5.4!fingertree_2.12.jar (386ms)
[info] Done updating.
[info] Updating ...
[info] Done updating.

@retronym
Copy link
Member

retronym commented Aug 27, 2019

Yes, the sun.* classes are absent from the ct.sym based archive of historical Java standard library APIs which {javac,scalac} -release use.

APIPhase, a custom compiler phase in SBT/Zinc that serializes the API of each file to perform change detection in incremental compilation, is attempting to serialize the API of the member @CallerSensitive ClassLoader getParent that de.sciss.synth.proc.impl.MemoryClassLoader inherits from j.l.ClassLoader. Until that point, scalac was doing okay without having a classfile for CallerSensitve -- it just used a StubSymbol in its place.

When APIPhase filters the list of annotations for those that extend scala.StaticAnnotation, the stub symbol gives up the ruse and aborts compilation. Static annotations are part of the API because the are could affect how client code is compiled.

I see two ways to make this work.

  • We could update StubSymbol.isStatic to return false, which should exclude that annotation from Zinc's API serialization.
  • We could update Zinc with effectively the same change.

/cc @jvican @eed3si9n

@retronym
Copy link
Member

An immediate workaround would be to declare an annotation sun.reflect.CallerSensitive in your project. You could structure the build in a way to avoid shipping this in your library.

@retronym
Copy link
Member

retronym commented Aug 27, 2019

I tried changing AnnotationInfo.isStatic. That makes the test project compile as I hoped.

$ sbt '++2.12.10-bin-b7c2be1-SNAPSHOT!' soundprocesses-core/compile
...
[info] Done compiling.
[success] Total time: 23 s, completed 27 Aug. 2019, 4:58:39 pm
commit b7c2be1df4fdc3d0664caaa88c82252d86531367 (HEAD -> ticket/11679, retronym/ticket/11679)
Author: Jason Zaugg <jzaugg@gmail.com>
Date:   Tue Aug 27 16:52:40 2019 +1000

    Assume StubSymbols are not StaticAnnotations

diff --git a/src/reflect/scala/reflect/internal/AnnotationInfos.scala b/src/reflect/scala/reflect/internal/AnnotationInfos.scala
index e438b2d677..619d9ac23e 100644
--- a/src/reflect/scala/reflect/internal/AnnotationInfos.scala
+++ b/src/reflect/scala/reflect/internal/AnnotationInfos.scala
@@ -323,7 +323,14 @@ trait AnnotationInfos extends api.Annotations { self: SymbolTable =>
     /** Check whether the type or any of the arguments are erroneous */
     def isErroneous = atp.isErroneous || args.exists(_.isErroneous)

-    def isStatic = symbol isNonBottomSubClass StaticAnnotationClass
+    def isStatic = {
+      symbol match {
+        case _: StubSymbol =>
+          false // See scala/bug#11679
+        case _ =>
+          symbol isNonBottomSubClass StaticAnnotationClass
+      }
+    }

     /** Check whether any of the arguments mention a symbol */
     def refsSymbol(sym: Symbol) = hasArgWhich(_.symbol == sym)

But I fear that this could make scalac too lenient -- when annotations based on stub symbols make it into signatures serialized by the pickler phase, we should emit a hard error.

@retronym
Copy link
Member

Extended patch to scalac that would retain the desired fail-fast behaviour in the pickler: retronym/scala#67

retronym added a commit to retronym/scala that referenced this issue Aug 28, 2019
In the reproduction in scala/bug#11679, an SBT build that uses
`--release 8` and as such does not have `sun._` on the classpath,
`APIPhase` (a custom compiler phase in SBT/Zinc that serializes the API
of each file to perform change detection in incremental compilation)
is attempting to serialize the API of the member
`@CallerSensitive ClassLoader getParent` that
`de.sciss.synth.proc.impl.MemoryClassLoader` inherits from
`j.l.ClassLoader`.

Until that point, scalac was doing okay without having a classfile
for `CallerSensitve` -- it just used a `StubSymbol` in its place.
Scala's pickle phase only serializes `.decls`, not `.members`.

When `APIPhase` filters the list of annotations:

  https://github.com/sbt/zinc/blob/4b414b6677/internal/compiler-bridge/src/main/scala/xsbt/ExtractAPI.scala#L789-L800

for those that extend `scala.StaticAnnotation`, the stub symbol fails
and aborts compilation. Static annotations are part of the API because
the are could affect how client code is compiled.

This commit changes `AnnotationInfo.isStatic` to return false
for annotations to absent classfiles. It also makes sure that we
still have a hard-error if annotations based on absent classfiles
are processed by the pickle phase.

I have manually tested this commit with the SBT project in the bug.

Fixes scala/bug#11679
@Sciss
Copy link
Author

Sciss commented Sep 23, 2019

Hello. Is there any workaround, I'm running into this again (cross building with Scala 2.12.10 and 2.13.1)

[error] /home/hhrutz/Documents/devel/dotterweide/core/src/main/scala/dotterweide/compiler/BytecodeInvoker.scala:49:44: Class sun.reflect.CallerSensitive not found - continuing with a stub.
[error]   private class DynamicClassLoader extends ClassLoader {
[error]                                            ^
[error] one error found

Is there a trick I can use? Unfortunately here I really have to compile under JDK 11 and I really need -release, 8, because one project module makes use of JDK 11 while all others should be compatible with JDK 8.

@lrytz
Copy link
Member

lrytz commented Sep 23, 2019

Maybe you could put the JDK 8 rt.jar on the Scala compiler's classpath while compiling that module (and then not use -release)?

@retronym
Copy link
Member

I have proposed a fix in SBT/Zinc: sbt/zinc#701

eed3si9n pushed a commit to eed3si9n/zinc that referenced this issue Oct 12, 2019
lrytz pushed a commit to lrytz/scala that referenced this issue Nov 5, 2019
@lrytz lrytz removed this from the 2.12.11 milestone Feb 27, 2020
jvican pushed a commit to scalacenter/zinc that referenced this issue Apr 27, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants