Error utilizing dependencies with scalac_plugin #4594

bhmiller · 2017-05-15T21:39:05Z

Tried 1.3.0.dev13 (first release I see allowing this syntax) and 1.3.0rc1 (and a few more in between):

Repro

Have all these in the same directory:

pants.ini:

[GLOBAL]
pants_version: 1.3.0rc1

BUILD:

scala_library(
    name='MyLibrary',
    sources=['MyClass.scala'],
    dependencies=[':MyPlugin'],
    scalac_plugins=['myPlugin'],
)

scalac_plugin(
    name='MyPlugin',
    sources=['MyPlugin.scala'],
    dependencies=[':MyJavaLibrary'],
#    java_sources=[':MyJavaLibrary'],
    classname='foo.MyPlugin',
    plugin='myPlugin',
)

java_library(
    name='MyJavaLibrary',
    sources=['MyJavaEnum.java'],
)

MyClass.scala:

package bar

object MyClass {
  val foo: String = "hello world"
}

MyJavaEnum.java:

package foo;

public enum MyJavaEnum {
    One, Two
}

MyPlugin.scala:

package foo

import tools.nsc.plugins.{Plugin, PluginComponent}
import tools.nsc.Global

class MyPlugin(val global: Global) extends Plugin {
  val name = "myPlugin"
  val description = "Test garbage plugin"
  val components = List[PluginComponent]()
  val enum = MyJavaEnum.One
}

If you run ./pants compile :: and you'll get:

17:30:32 00:02   [compile]
...
17:30:32 00:02     [compile]
17:30:32 00:02     [zinc]
17:30:32 00:02       [cache]   
                   No cached artifacts for 3 targets.
                   Invalidated 3 targets.
17:30:32 00:02       [isolation-zinc-pool-bootstrap] 
                   [1/3] Compiling 1 zinc source in 1 target (//:MyJavaLibrary).
17:30:32 00:02       [compile]
...
17:31:32 01:02         [zinc]
                       [info] Compiling 1 Java source to /<dir>/.pants.d/compile/zinc/c95338cea9cc/.MyJavaLibrary/current/classes...
                       [info] Compile success at May 15, 2017 5:31:33 PM [1.391s]

                   [2/3] Compiling 1 zinc source in 1 target (//:MyPlugin).
17:31:33 01:03       [compile]
                     
17:31:33 01:03         [zinc]
                       [info] Compiling 1 Scala source to /<dir>/.pants.d/compile/zinc/c95338cea9cc/.MyPlugin/current/classes...
                       [info] Compile success at May 15, 2017 5:31:39 PM [6.151s]
                        
                   [3/3] Compiling 1 zinc source in 1 target (//:MyLibrary).
17:31:39 01:09       [compile]
                     
17:31:40 01:10         [zinc]
                       [info] Compiling 1 Scala source to /<dir>/.pants.d/compile/zinc/c95338cea9cc/.MyLibrary/current/classes...
                       [error] ## Exception when compiling /<dir>/MyClass.scala and others...
                       [error] null
                       [error] sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                       [error] sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                       [error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                       [error] java.lang.reflect.Method.invoke(Method.java:498)
                       [error] xsbt.CachedCompiler0$Compiler.superCall(CompilerInterface.scala:233)
                       [error] xsbt.CachedCompiler0$Compiler.superComputePhaseDescriptors(CompilerInterface.scala:226)
                       [error] xsbt.CachedCompiler0$Compiler.phaseDescriptors$lzycompute(CompilerInterface.scala:223)
                       [error] xsbt.CachedCompiler0$Compiler.phaseDescriptors(CompilerInterface.scala:218)
                       [error] scala.tools.nsc.Global$Run.<init>(Global.scala:1171)
                       [error] xsbt.CachedCompiler0$$anon$2.<init>(CompilerInterface.scala:119)
                       [error] xsbt.CachedCompiler0.run(CompilerInterface.scala:119)
                       [error] xsbt.CachedCompiler0.run(CompilerInterface.scala:108)
                       [error] xsbt.CompilerInterface.run(CompilerInterface.scala:26)
                       [error] sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                       [error] sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                       [error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                       [error] java.lang.reflect.Method.invoke(Method.java:498)
                       [error] __shaded_by_pants__.sbt.internal.inc.AnalyzingCompiler.call(AnalyzingCompiler.scala:107)
                       [error] __shaded_by_pants__.sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:42)
                       [error] __shaded_by_pants__.sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:35)
                       [error] __shaded_by_pants__.sbt.internal.inc.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply$mcV$sp(MixedAnalyzingCompiler.scala:50)
                       [error] __shaded_by_pants__.sbt.internal.inc.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply(MixedAnalyzingCompiler.scala:50)
                       [error] __shaded_by_pants__.sbt.internal.inc.MixedAnalyzingCompiler$$anonfun$compileScala$1$1.apply(MixedAnalyzingCompiler.scala:50)
                       [error] __shaded_by_pants__.sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:74)
                       [error] __shaded_by_pants__.sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:49)
                       [error] __shaded_by_pants__.sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:64)
                       [error] __shaded_by_pants__.sbt.internal.inc.IncrementalCompilerImpl$$anonfun$compileInternal$1.apply(IncrementalCompilerImpl.scala:132)
                       [error] __shaded_by_pants__.sbt.internal.inc.IncrementalCompilerImpl$$anonfun$compileInternal$1.apply(IncrementalCompilerImpl.scala:132)
                       [error] __shaded_by_pants__.sbt.internal.inc.Incremental$.doCompile(Incremental.scala:95)
                       [error] __shaded_by_pants__.sbt.internal.inc.Incremental$$anonfun$1$$anonfun$apply$1.apply(Incremental.scala:79)
                       [error] __shaded_by_pants__.sbt.internal.inc.Incremental$$anonfun$1$$anonfun$apply$1.apply(Incremental.scala:79)
                       [error] __shaded_by_pants__.sbt.internal.inc.IncrementalCommon.recompileClasses(IncrementalCommon.scala:70)
                       [error] __shaded_by_pants__.sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:39)
                       [error] __shaded_by_pants__.sbt.internal.inc.Incremental$$anonfun$1.apply(Incremental.scala:78)
                       [error] __shaded_by_pants__.sbt.internal.inc.Incremental$$anonfun$1.apply(Incremental.scala:77)
                       [error] __shaded_by_pants__.sbt.internal.inc.Incremental$.manageClassfiles(Incremental.scala:122)
                       [error] __shaded_by_pants__.sbt.internal.inc.Incremental$.compile(Incremental.scala:77)
                       [error] __shaded_by_pants__.sbt.internal.inc.IncrementalCompile$.apply(Compile.scala:60)
                       [error] __shaded_by_pants__.sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:132)
                       [error] __shaded_by_pants__.sbt.internal.inc.IncrementalCompilerImpl.incrementalCompile(IncrementalCompilerImpl.scala:87)
                       [error] org.pantsbuild.zinc.Compiler.compile(Compiler.scala:203)
                       [error] org.pantsbuild.zinc.Main$.run(Main.scala:97)
                       [error] org.pantsbuild.zinc.Main$.main(Main.scala:16)
                       [error] org.pantsbuild.zinc.Main.main(Main.scala)
                       [error] sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                       [error] sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
                       [error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
                       [error] java.lang.reflect.Method.invoke(Method.java:498)
                       [error] com.martiansoftware.nailgun.NGSession.run(NGSession.java:280)
                       [error]          
                       
                   compile(//:MyLibrary) failed: Zinc compile failed.
FAILURE: Compilation failure: Failed jobs: compile(//:MyLibrary)


               Waiting for background workers to finish.
17:31:40 01:10   [complete]

If I change the BUILD file to use java_sources instead of dependencies, then everything is hunky dory.

Also FYI in case it matters, for my actual error vs. the repro above, I get a slightly different error: java.lang.NoClassDefFoundError: foo/MyJavaEnum. If you think it's worth it, I can try and improve the repro to get that error, but the work around is the same for both the simplified repro and my actual use case.

The text was updated successfully, but these errors were encountered:

stuhood · 2017-05-15T21:53:02Z

Thanks for the report!

It sounds like the dependencies of the plugin are not being included on the classpath when it is used as a plugin. The reason java_sources would work around this is that it has the effect of merging the two targets it links (from a compilation perspective).

bhmiller · 2017-05-15T23:06:17Z

Yeah, work around means less urgent for me to get the fix. Getting through a couple things I've been meaning to try and repro before I completely miss the boat on 1.3.0.

stuhood · 2017-05-18T23:49:08Z

Since this is not a regression, but a missing bit in a new feature, I'm going to drop it from the 1.3.0 milestone. Should be straightforward to fix for 1.3.1.

bhmiller · 2017-06-07T22:19:33Z

thoughts on tagging this to the 1.3.1 milestone now that 1.3.0 is done? thanks.

stuhood · 2017-06-07T23:34:33Z

Yep.

stuhood · 2017-06-07T23:37:40Z

@bhmiller : If you have time to tackle this one (possibly with @benjyw 's assistance), it would be much appreciated.

bhmiller · 2017-06-08T15:22:29Z

Sure can give a whirl. What turn around timeline are you looking for? Caveat emptor, I don't do much python, so might not be the quickest turn around.

benjyw · 2017-10-03T18:51:39Z

Doing some inbox cleaning and found this, so checking in on status. Still an issue?

stuhood · 2017-10-03T18:55:25Z

@benjyw : AFAIK, yes.

benjyw · 2017-10-17T03:00:11Z

So far I've determined that generally the dependencies of a plugin are on the classpath when it's used as a plugin. Just not when the plugin is scala and the dep is java. Still figuring out why.

benjyw · 2017-10-17T03:04:10Z

Huh. The Java classes are on the classpath when the error occurs, so something else is going on.

benjyw · 2017-10-17T03:15:19Z

OK, so the zinc command lines in the java_sources case vs. the dependencies case are identical, except that the java_sources case has the java classes in the same /classes dir as the scala classes, and does not have the java /classes dir on the classpath. The dependencies case has those two in separate dirs, but both are on the classpath.

benjyw · 2017-10-17T03:26:27Z

Hmm, so this has nothing to do with the dep being Java. It reproduces even with a Scala dep. Oddly enough, the java_sources workaround still works when the "java sources" are actually a scala_library with .scala sources...

So it boils down to zinc not loading plugin deps from outside the main plugin class's classpath element.

benjyw · 2017-10-17T03:28:58Z

Sure enough, this is a limitation of scalac: https://groups.google.com/forum/#!topic/sbt-dev/ztKxUpxM5pE

benjyw · 2017-10-17T05:37:43Z

The solution is to modify ZincCompile to add the dependency classpath elements to the -Xplugin flag: -S-Xplugin:<classpath element containing plugin metadata>:<classpath element for dep1>:...

My experiments show that scalac will load the plugin from the first element (and not from any subsequent elements, if they happen to also be plugins, which is good) but will load deps from any element.

benjyw · 2017-10-17T08:48:44Z

Addressed in #4987

bhmiller · 2017-10-18T16:00:00Z

awesome. thanks for sorting through this.

stuhood added bug jvm needs-cherrypick labels May 15, 2017

stuhood added this to the 1.3.0 milestone May 15, 2017

stuhood removed this from the 1.3.0 milestone May 18, 2017

stuhood removed the needs-cherrypick label May 19, 2017

stuhood added this to the 1.3.1 milestone Jun 7, 2017

benjyw mentioned this issue Oct 17, 2017

Allow in-repo scalac plugins to have in-repo deps. #4987

Merged

benjyw closed this as completed Oct 18, 2017

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Error utilizing dependencies with scalac_plugin #4594

Error utilizing dependencies with scalac_plugin #4594

bhmiller commented May 15, 2017 •

edited

stuhood commented May 15, 2017

bhmiller commented May 15, 2017

stuhood commented May 18, 2017

bhmiller commented Jun 7, 2017

stuhood commented Jun 7, 2017

stuhood commented Jun 7, 2017

bhmiller commented Jun 8, 2017

benjyw commented Oct 3, 2017

stuhood commented Oct 3, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017 •

edited

benjyw commented Oct 17, 2017

bhmiller commented Oct 18, 2017

Error utilizing dependencies with scalac_plugin #4594

Error utilizing dependencies with scalac_plugin #4594

Comments

bhmiller commented May 15, 2017 • edited

Repro

stuhood commented May 15, 2017

bhmiller commented May 15, 2017

stuhood commented May 18, 2017

bhmiller commented Jun 7, 2017

stuhood commented Jun 7, 2017

stuhood commented Jun 7, 2017

bhmiller commented Jun 8, 2017

benjyw commented Oct 3, 2017

stuhood commented Oct 3, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017

benjyw commented Oct 17, 2017 • edited

benjyw commented Oct 17, 2017

bhmiller commented Oct 18, 2017

bhmiller commented May 15, 2017 •

edited

benjyw commented Oct 17, 2017 •

edited