This repository has been archived by the owner. It is now read-only.

zinc doesn't seems to work in multithreading environment #31

Closed
jakozaur opened this Issue Jul 30, 2013 · 8 comments

Comments

Projects
None yet
2 participants
@jakozaur

jakozaur commented Jul 30, 2013

I am using zinc 0.2.5 with scala-maven-plugin and using maven parallel build.

The build is successful most of the times, but sometimes fails, e.g. with [1] or couldn't find dependency libraries [2].

Seems like the root cause is that Sun Java compiler is not thread safe, despite claiming so (there are some bugs on web with similar symptom, e.g. https://code.google.com/p/maven-annotation-plugin/issues/detail?id=32 ).

Can we add an argument to zinc which will cause to always fork/not reuse at the same java compiler? It is unlikely that Java compiler will get fixed in short time, but that's how maven-compiler-plugin deal with it (http://maven.apache.org/plugins/maven-compiler-plugin/compile-mojo.html#fork ).

Many thanks,
Jacek

[1]
[error] An exception has occurred in the compiler (1.6.0_45). Please file a bug at the Java Developer Connection (http://java.sun.com/webapps/bugreport) after checking the Bug Parade for duplicates. Include your program and the following diagnostic in your report. Thank you.
[error] java.lang.NullPointerException
[error] at com.sun.tools.javac.jvm.ClassReader.enterClass(ClassReader.java:1740)
[error] at com.sun.tools.javac.jvm.ClassReader.enterClass(ClassReader.java:1755)
[error] at com.sun.tools.javac.code.Symtab.enterClass(Symtab.java:260)
[error] at com.sun.tools.javac.code.Symtab.(Symtab.java:416)
[error] at com.sun.tools.javac.code.Symtab.instance(Symtab.java:45)
[error] at com.sun.tools.javac.jvm.ClassReader.(ClassReader.java:205)
[error] at com.sun.tools.javac.jvm.ClassReader.instance(ClassReader.java:168)
[error] at com.sun.tools.javac.main.JavaCompiler.(JavaCompiler.java:293)
[error] at com.sun.tools.javac.main.JavaCompiler.instance(JavaCompiler.java:72)
[error] at com.sun.tools.javac.main.Main.compile(Main.java:340)
[error] at com.sun.tools.javac.main.Main.compile(Main.java:279)
[error] at com.sun.tools.javac.main.Main.compile(Main.java:270)
[error] at com.sun.tools.javac.Main.compile(Main.java:87)
[error] at sun.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
[error] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[error] at java.lang.reflect.Method.invoke(Method.java:597)
[error] at sbt.compiler.JavaCompiler$$anon$2.exec(JavaCompiler.scala:40)
[error] at sbt.compiler.JavaCompiler$$anonfun$2.apply(JavaCompiler.scala:93)
[error] at sbt.compiler.JavaCompiler$$anonfun$2.apply(JavaCompiler.scala:85)
[error] at sbt.compiler.JavaCompiler$$anonfun$directOrForkJavac$1.apply(JavaCompiler.scala:72)
[error] at sbt.compiler.JavaCompiler$$anonfun$directOrForkJavac$1.apply(JavaCompiler.scala:71)
[error] at sbt.compiler.JavaCompiler$$anon$1.compile(JavaCompiler.scala:57)
[error] at sbt.compiler.JavaTool$class.apply(JavaCompiler.scala:26)
[error] at sbt.compiler.JavaCompiler$$anon$1.apply(JavaCompiler.scala:51)
[error] at sbt.compiler.JavaCompiler$class.compile(JavaCompiler.scala:17)
[error] at sbt.compiler.JavaCompiler$$anon$1.compile(JavaCompiler.scala:51)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4$$anonfun$compileJava$1$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply$mcV$sp(AggressiveCompile.scala:83)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4$$anonfun$compileJava$1$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply(AggressiveCompile.scala:83)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4$$anonfun$compileJava$1$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$2.apply(AggressiveCompile.scala:83)
[error] at sbt.compiler.AggressiveCompile.sbt$compiler$AggressiveCompile$$timed(AggressiveCompile.scala:101)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4$$anonfun$compileJava$1$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(AggressiveCompile.scala:82)
[error] at sbt.classfile.Analyze$.apply(Analyze.scala:85)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4$$anonfun$compileJava$1$1.apply$mcV$sp(AggressiveCompile.scala:81)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4$$anonfun$compileJava$1$1.apply(AggressiveCompile.scala:81)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4$$anonfun$compileJava$1$1.apply(AggressiveCompile.scala:81)
[error] at sbt.compiler.AggressiveCompile.sbt$compiler$AggressiveCompile$$timed(AggressiveCompile.scala:101)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4.compileJava$1(AggressiveCompile.scala:80)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4.apply(AggressiveCompile.scala:88)
[error] at sbt.compiler.AggressiveCompile$$anonfun$4.apply(AggressiveCompile.scala:60)
[error] at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:24)
[error] at sbt.inc.IncrementalCompile$$anonfun$doCompile$1.apply(Compile.scala:22)
[error] at sbt.inc.Incremental$.cycle(Incremental.scala:52)
[error] at sbt.inc.Incremental$.compile(Incremental.scala:29)
[error] at sbt.inc.IncrementalCompile$.apply(Compile.scala:20)
[error] at sbt.compiler.AggressiveCompile.compile2(AggressiveCompile.scala:96)
[error] at sbt.compiler.AggressiveCompile.compile1(AggressiveCompile.scala:44)
[error] at com.typesafe.zinc.Compiler.compile(Compiler.scala:158)
[error] at com.typesafe.zinc.Main$.run(Main.scala:95)
[error] at com.typesafe.zinc.Nailgun$.zinc(Nailgun.scala:93)
[error] at com.typesafe.zinc.Nailgun$.nailMain(Nailgun.scala:82)
[error] at com.typesafe.zinc.Nailgun.nailMain(Nailgun.scala)
[error] at sun.reflect.GeneratedMethodAccessor2.invoke(Unknown Source)
[error] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
[error] at java.lang.reflect.Method.invoke(Method.java:597)
[error] at com.martiansoftware.nailgun.NGSession.run(NGSession.java:280)
[error] Compile failed at Jul 29, 2013 11:52:07 PM [13.693s]

[2]
[INFO] Building jar: /Users/jacek/Development/sumo/ops/target/ops-20.1-SNAPSHOT.jar
[error] /Users/jacek/Development/sumo/bill/target/generated-sources/cxf/com/sumologic/zuora/generated/SubscribeResult.java:6: package javax.imp.bind does not exist
[error] import javax.xml.bind.JAXBElement;

@pvlugter

This comment has been minimized.

Show comment
Hide comment
@pvlugter

pvlugter Aug 1, 2013

Member

Sure, forking javac is straightforward. Zinc/sbt already fork javac when java-home is defined. We can add another option to always fork, whether or not java-home is set.

And we'll need to add this option to the scala maven plugin.

Member

pvlugter commented Aug 1, 2013

Sure, forking javac is straightforward. Zinc/sbt already fork javac when java-home is defined. We can add another option to always fork, whether or not java-home is set.

And we'll need to add this option to the scala maven plugin.

@ghost ghost assigned pvlugter Aug 1, 2013

@jakozaur

This comment has been minimized.

Show comment
Hide comment
@jakozaur

jakozaur Aug 1, 2013

@pvlugter Thank so much for your reply.

I just checked on my machine on JAVA HOME is not set:
jacek@jaceks-macbookpro:~/Development/sumo/sla master*$ zinc -status
Nailgun server running with 1 cached compiler

Version = 0.2.5

Zinc compiler cache limit = 5
Resident scalac cache limit = 0
Analysis cache limit = 5

Compiler(Scala 2.10.0) [3005d89e]
Setup = {
scala compiler = /Users/jacek/.m2/repository/org/scala-lang/scala-compiler/2.10.0/scala-compiler-2.10.0.jar
scala library = /Users/jacek/.m2/repository/org/scala-lang/scala-library/2.10.0/scala-library-2.10.0.jar
scala extra = {
/Users/jacek/.m2/repository/org/scala-lang/scala-reflect/2.10.0/scala-reflect-2.10.0.jar
}
sbt interface = /usr/local/Cellar/zinc/0.2.5/libexec/lib/sbt-interface.jar
compiler interface sources = /usr/local/Cellar/zinc/0.2.5/libexec/lib/compiler-interface-sources.jar
java home =
cache directory = /Users/jacek/.zinc/0.2.5
}

Will set it and see if that helped.

jakozaur commented Aug 1, 2013

@pvlugter Thank so much for your reply.

I just checked on my machine on JAVA HOME is not set:
jacek@jaceks-macbookpro:~/Development/sumo/sla master*$ zinc -status
Nailgun server running with 1 cached compiler

Version = 0.2.5

Zinc compiler cache limit = 5
Resident scalac cache limit = 0
Analysis cache limit = 5

Compiler(Scala 2.10.0) [3005d89e]
Setup = {
scala compiler = /Users/jacek/.m2/repository/org/scala-lang/scala-compiler/2.10.0/scala-compiler-2.10.0.jar
scala library = /Users/jacek/.m2/repository/org/scala-lang/scala-library/2.10.0/scala-library-2.10.0.jar
scala extra = {
/Users/jacek/.m2/repository/org/scala-lang/scala-reflect/2.10.0/scala-reflect-2.10.0.jar
}
sbt interface = /usr/local/Cellar/zinc/0.2.5/libexec/lib/sbt-interface.jar
compiler interface sources = /usr/local/Cellar/zinc/0.2.5/libexec/lib/compiler-interface-sources.jar
java home =
cache directory = /Users/jacek/.zinc/0.2.5
}

Will set it and see if that helped.

@jakozaur

This comment has been minimized.

Show comment
Hide comment
@jakozaur

jakozaur Aug 1, 2013

Not sure if it's related, but I hit also error where dependency is not found:
[INFO] Using zinc server for incremental compilation
[info] Compiling 10 Scala sources and 5 Java sources to /Users/jakozaur/Work/SumoLogic/sumo/cq-client/target/classes...
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ concierge ---
[INFO] Deleting /Users/jakozaur/Work/SumoLogic/sumo/concierge/target
[INFO] Deleting /Users/jakozaur/Work/SumoLogic/sumo/ops/assemblies/latest (includes = [concierge-, concierge-/], excludes = [])
[error] /Users/jakozaur/Work/SumoLogic/sumo/cq-client/src/main/scala/com/sumologic/scala/cq/client/ContinuousQueryManager.scala:5: object util is not a member of package com.sumologic
[error] import com.sumologic.util.concurrent.FutureResult
...

It happens only when zinc is used.

jakozaur commented Aug 1, 2013

Not sure if it's related, but I hit also error where dependency is not found:
[INFO] Using zinc server for incremental compilation
[info] Compiling 10 Scala sources and 5 Java sources to /Users/jakozaur/Work/SumoLogic/sumo/cq-client/target/classes...
[INFO]
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ concierge ---
[INFO] Deleting /Users/jakozaur/Work/SumoLogic/sumo/concierge/target
[INFO] Deleting /Users/jakozaur/Work/SumoLogic/sumo/ops/assemblies/latest (includes = [concierge-, concierge-/], excludes = [])
[error] /Users/jakozaur/Work/SumoLogic/sumo/cq-client/src/main/scala/com/sumologic/scala/cq/client/ContinuousQueryManager.scala:5: object util is not a member of package com.sumologic
[error] import com.sumologic.util.concurrent.FutureResult
...

It happens only when zinc is used.

@pvlugter

This comment has been minimized.

Show comment
Hide comment
@pvlugter

pvlugter Aug 1, 2013

Member

Hi Jacek. Can you put together a sample project with these problems?

Member

pvlugter commented Aug 1, 2013

Hi Jacek. Can you put together a sample project with these problems?

@jakozaur

This comment has been minimized.

Show comment
Hide comment
@jakozaur

jakozaur Aug 2, 2013

Hi Peter,

Unfortunately rather not, it is a code of my employer (Sumo Logic). Moreover, it's fairly large (~100 modules) and sometime I have to build a few times before I hit the issue.

I would rather spent some time to isolate/replicate that issue on smaller project.

Many thanks,
Jacek

jakozaur commented Aug 2, 2013

Hi Peter,

Unfortunately rather not, it is a code of my employer (Sumo Logic). Moreover, it's fairly large (~100 modules) and sometime I have to build a few times before I hit the issue.

I would rather spent some time to isolate/replicate that issue on smaller project.

Many thanks,
Jacek

@pvlugter

This comment has been minimized.

Show comment
Hide comment
@pvlugter

pvlugter Aug 2, 2013

Member

If it can be replicated in a small sample project, that would be great (I don't mean you should include any code from your work project).

Member

pvlugter commented Aug 2, 2013

If it can be replicated in a small sample project, that would be great (I don't mean you should include any code from your work project).

@pvlugter

This comment has been minimized.

Show comment
Hide comment
@pvlugter

pvlugter Sep 5, 2013

Member

Sorry about delay. Zinc 0.3.0 has been published, which includes a -fork-java option.

The scala maven plugin will need to be updated to use this. I'm sure that contribution will be much appreciated if you'd like to do that.

Member

pvlugter commented Sep 5, 2013

Sorry about delay. Zinc 0.3.0 has been published, which includes a -fork-java option.

The scala maven plugin will need to be updated to use this. I'm sure that contribution will be much appreciated if you'd like to do that.

@pvlugter pvlugter closed this Sep 5, 2013

@jakozaur

This comment has been minimized.

Show comment
Hide comment
@jakozaur

jakozaur Sep 6, 2013

@pvlugter thank you so much! Will try with that option and see if that helps.

jakozaur commented Sep 6, 2013

@pvlugter thank you so much! Will try with that option and see if that helps.

retronym pushed a commit to retronym/zinc-legacy that referenced this issue Nov 1, 2017

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.