Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

for-yield comprehension or flatMap gives an IOException: error: Operation not permitted #9132

Closed
scabug opened this issue Feb 2, 2015 · 10 comments
Closed

for-yield comprehension or flatMap gives an IOException: error: Operation not permitted #9132

scabug opened this issue Feb 2, 2015 · 10 comments
Assignees
@retronym
Labels
compiler crash

Comments

@scabug
Copy link

scabug commented Feb 2, 2015

When I try to compile following code, with a for-comprehension of mostly Try (tries). UploadResult is a case class

for {
     _ <- markerUtil.uploadingStarted(source, destPath, files.map(_.path)) // Try[Unit]
     inputStream <- createInputStream(files) // Try[InputStream]
     _ <- store.upload(destPath, inputStream) // Try[Unit]
     _ <- markerUtil.uploadingDone(source) // Try[Unit]
     _ <- deleteFiles(files) // Try[Unit]
     _ <- markerUtil.uploadedDone(source) // Try[Unit]
   } yield ()

I get following output. Strangely the error happens in a completely different part of the code (in another package).

[INFO] /vagrant/core-compute-jobs-factstore/src/main/scala:-1: info: compiling
[INFO] Compiling 14 source files to /vagrant/core-compute-jobs-factstore/target/classes at 1422892024304
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[ERROR] error:
[INFO]      while compiling: /vagrant/core-compute-jobs-factstore/src/main/scala/com/example/core/compute/jobs/factstore/util/FileSystemFactory.scala
[INFO]         during phase: jvm
[INFO]      library version: version 2.10.4
[INFO]     compiler version: version 2.10.4
[INFO]   reconstructed args: -classpath /vagrant/core-compute-jobs-factstore/target/classes:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-client/2.0.0-mr1-cdh4.4.0/hadoop-client-2.0.0-mr1-cdh4.4.0.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-common/2.0.0-cdh4.4.0/hadoop-common-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-annotations/2.0.0-cdh4.4.0/hadoop-annotations-2.0.0-cdh4.4.0.jar:/usr/lib/jvm/java-7-openjdk-amd64/jre/../lib/tools.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/home/vagrant/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/home/vagrant/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/home/vagrant/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/home/vagrant/.m2/repository/commons-lang/commons-lang/2.5/commons-lang-2.5.jar:/home/vagrant/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/home/vagrant/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/home/vagrant/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/home/vagrant/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/home/vagrant/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/home/vagrant/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:/home/vagrant/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:/home/vagrant/.m2/repository/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5.jar:/home/vagrant/.m2/repository/org/apache/avro/avro/1.7.4/avro-1.7.4.jar:/home/vagrant/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/home/vagrant/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/home/vagrant/.m2/repository/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-auth/2.0.0-cdh4.4.0/hadoop-auth-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/com/jcraft/jsch/0.1.42/jsch-0.1.42.jar:/home/vagrant/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5-cdh4.4.0/zookeeper-3.4.5-cdh4.4.0.jar:/home/vagrant/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.0.0-cdh4.4.0/hadoop-hdfs-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar:/home/vagrant/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar:/home/vagrant/.m2/repository/asm/asm/3.1/asm-3.1.jar:/home/vagrant/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-core/2.0.0-mr1-cdh4.4.0/hadoop-core-2.0.0-mr1-cdh4.4.0.jar:/home/vagrant/.m2/repository/hsqldb/hsqldb/1.8.0.10/hsqldb-1.8.0.10.jar:/home/vagrant/.m2/repository/com/amazonaws/aws-java-sdk/1.6.4/aws-java-sdk-1.6.4.jar:/home/vagrant/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/home/vagrant/.m2/repository/commons-codec/commons-codec/1.3/commons-codec-1.3.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.1.1/jackson-core-2.1.1.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.1.1/jackson-databind-2.1.1.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.1.1/jackson-annotations-2.1.1.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar:/home/vagrant/.m2/repository/org/apache/spark/spark-core_2.10/1.1.1/spark-core_2.10-1.1.1.jar:/home/vagrant/.m2/repository/net/java/dev/jets3t/jets3t/0.7.1/jets3t-0.7.1.jar:/home/vagrant/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-recipes/2.4.0/curator-recipes-2.4.0.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-framework/2.4.0/curator-framework-2.4.0.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-client/2.4.0/curator-client-2.4.0.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-plus/8.1.14.v20131031/jetty-plus-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.transaction/1.1.1.v201105210645/javax.transaction-1.1.1.v201105210645.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-webapp/8.1.14.v20131031/jetty-webapp-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-xml/8.1.14.v20131031/jetty-xml-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-servlet/8.1.14.v20131031/jetty-servlet-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-jndi/8.1.14.v20131031/jetty-jndi-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.mail.glassfish/1.4.1.v201005082020/javax.mail.glassfish-1.4.1.v201005082020.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.activation/1.1.0.v201105071233/javax.activation-1.1.0.v201105071233.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-security/8.1.14.v20131031/jetty-security-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-util/8.1.14.v20131031/jetty-util-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-server/8.1.14.v20131031/jetty-server-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-continuation/8.1.14.v20131031/jetty-continuation-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-http/8.1.14.v20131031/jetty-http-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-io/8.1.14.v20131031/jetty-io-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/com/google/guava/guava/14.0.1/guava-14.0.1.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/home/vagrant/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/home/vagrant/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar:/home/vagrant/.m2/repository/org/slf4j/jul-to-slf4j/1.7.5/jul-to-slf4j-1.7.5.jar:/home/vagrant/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.5/jcl-over-slf4j-1.7.5.jar:/home/vagrant/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/home/vagrant/.m2/repository/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar:/home/vagrant/.m2/repository/com/ning/compress-lzf/1.0.0/compress-lzf-1.0.0.jar:/home/vagrant/.m2/repository/org/xerial/snappy/snappy-java/1.0.5.3/snappy-java-1.0.5.3.jar:/home/vagrant/.m2/repository/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar:/home/vagrant/.m2/repository/com/twitter/chill_2.10/0.3.6/chill_2.10-0.3.6.jar:/home/vagrant/.m2/repository/com/twitter/chill-java/0.3.6/chill-java-0.3.6.jar:/home/vagrant/.m2/repository/commons-net/commons-net/2.2/commons-net-2.2.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-remote_2.10/2.2.3-shaded-protobuf/akka-remote_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-actor_2.10/2.2.3-shaded-protobuf/akka-actor_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/io/netty/netty/3.6.6.Final/netty-3.6.6.Final.jar:/home/vagrant/.m2/repository/org/spark-project/protobuf/protobuf-java/2.4.1-shaded/protobuf-java-2.4.1-shaded.jar:/home/vagrant/.m2/repository/org/uncommons/maths/uncommons-maths/1.2.2a/uncommons-maths-1.2.2a.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-slf4j_2.10/2.2.3-shaded-protobuf/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/org/json4s/json4s-jackson_2.10/3.2.10/json4s-jackson_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/json4s/json4s-core_2.10/3.2.10/json4s-core_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/json4s/json4s-ast_2.10/3.2.10/json4s-ast_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/scala-lang/scalap/2.10.0/scalap-2.10.0.jar:/home/vagrant/.m2/repository/colt/colt/1.2.0/colt-1.2.0.jar:/home/vagrant/.m2/repository/concurrent/concurrent/1.3.4/concurrent-1.3.4.jar:/home/vagrant/.m2/repository/org/apache/mesos/mesos/0.18.1/mesos-0.18.1-shaded-protobuf.jar:/home/vagrant/.m2/repository/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar:/home/vagrant/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-core/3.0.0/metrics-core-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-jvm/3.0.0/metrics-jvm-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-json/3.0.0/metrics-json-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-graphite/3.0.0/metrics-graphite-3.0.0.jar:/home/vagrant/.m2/repository/org/tachyonproject/tachyon-client/0.5.0/tachyon-client-0.5.0.jar:/home/vagrant/.m2/repository/org/tachyonproject/tachyon/0.5.0/tachyon-0.5.0.jar:/home/vagrant/.m2/repository/org/spark-project/pyrolite/2.0.1/pyrolite-2.0.1.jar:/home/vagrant/.m2/repository/net/sf/py4j/py4j/0.8.2.1/py4j-0.8.2.1.jar:/home/vagrant/.m2/repository/joda-time/joda-time/2.2/joda-time-2.2.jar:/home/vagrant/.m2/repository/org/apache/httpcomponents/httpclient/4.2.6/httpclient-4.2.6.jar:/home/vagrant/.m2/repository/org/apache/httpcomponents/httpcore/4.2.5/httpcore-4.2.5.jar:/home/vagrant/.m2/repository/com/esotericsoftware/kryo/kryo/2.22/kryo-2.22.jar:/home/vagrant/.m2/repository/com/typesafe/config/1.2.1/config-1.2.1.jar:/home/vagrant/.m2/repository/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/core-common-datatypes-domain-1.6.0-SNAPSHOT.jar:/home/vagrant/.m2/repository/org/apache/thrift/libthrift/0.9.1/libthrift-0.9.1.jar:/home/vagrant/.m2/repository/com/twitter/scrooge-core_2.10/3.12.3/scrooge-core_2.10-3.12.3.jar:/home/vagrant/.m2/repository/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/core-compute-util-spark-metrics-1.0.0-SNAPSHOT.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-ganglia/3.0.1/metrics-ganglia-3.0.1.jar:/home/vagrant/.m2/repository/info/ganglia/gmetric4j/gmetric4j/1.0.3/gmetric4j-1.0.3.jar:/home/vagrant/.m2/repository/org/acplt/oncrpc/1.0.7/oncrpc-1.0.7.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-reflect/2.10.3/scala-reflect-2.10.3.jar:/home/vagrant/.m2/repository/junit/junit/4.11/junit-4.11.jar:/home/vagrant/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-compiler/2.10.3/scala-compiler-2.10.3.jar -d /vagrant/core-compute-jobs-factstore/target/classes
[INFO]
[INFO]   last tree to typer: Literal(Constant(com.example.core.compute.jobs.factstore.source.SourceDirectory))
[INFO]               symbol: null
[INFO]    symbol definition: null
[INFO]                  tpe: Class(classOf[com.example.core.compute.jobs.factstore.source.SourceDirectory])
[INFO]        symbol owners:
[INFO]       context owners: class FileSystemFactory -> package util
[INFO]
[INFO] == Enclosing template or block ==
[INFO]
[INFO] Template( // val <local FileSystemFactory>: <notype> in class FileSystemFactory, tree.tpe=com.example.core.compute.jobs.factstore.util.FileSystemFactory
[INFO]   "java.lang.Object", "scala.Serializable" // parents
[INFO]   ValDef(
[INFO]     private
[INFO]     "_"
[INFO]     <tpt>
[INFO]     <empty>
[INFO]   )
[INFO]   // 3 statements
[INFO]   ValDef( // private[this] val uri: String in class FileSystemFactory
[INFO]     private <local> <paramaccessor> <triedcooking>
[INFO]     "uri"
[INFO]     <tpt> // tree.tpe=String
[INFO]     <empty>
[INFO]   )
[INFO]   DefDef( // def get(): org.apache.hadoop.fs.FileSystem in class FileSystemFactory
[INFO]     <method> <triedcooking>
[INFO]     "get"
[INFO]     []
[INFO]     List(Nil)
[INFO]     <tpt> // tree.tpe=org.apache.hadoop.fs.FileSystem
[INFO]     Apply( // def get(x$1: java.net.URI,x$2: org.apache.hadoop.conf.Configuration): org.apache.hadoop.fs.FileSystem in object FileSystem, tree.tpe=org.apache.hadoop.fs.FileSystem
[INFO]       "org"."apache"."hadoop"."fs"."FileSystem"."get" // def get(x$1: java.net.URI,x$2: org.apache.hadoop.conf.Configuration): org.apache.hadoop.fs.FileSystem in object FileSystem, tree.tpe=(x$1: java.net.URI, x$2: org.apache.hadoop.conf.Configuration)org.apache.hadoop.fs.FileSystem
[INFO]       // 2 arguments
[INFO]       Apply( // def <init>(x$1: String): java.net.URI in class URI, tree.tpe=java.net.URI
[INFO]         new java.net.URI."<init>" // def <init>(x$1: String): java.net.URI in class URI, tree.tpe=(x$1: String)java.net.URI
[INFO]         FileSystemFactory.this."uri" // private[this] val uri: String in class FileSystemFactory, tree.tpe=String
[INFO]       )
[INFO]       Apply( // def <init>(): org.apache.hadoop.conf.Configuration in class Configuration, tree.tpe=org.apache.hadoop.conf.Configuration
[INFO]         new org.apache.hadoop.conf.Configuration."<init>" // def <init>(): org.apache.hadoop.conf.Configuration in class Configuration, tree.tpe=()org.apache.hadoop.conf.Configuration
[INFO]         Nil
[INFO]       )
[INFO]     )
[INFO]   )
[INFO]   DefDef( // def <init>(uri: String): com.example.core.compute.jobs.factstore.util.FileSystemFactory in class FileSystemFactory
[INFO]     <method> <triedcooking>
[INFO]     "<init>"
[INFO]     []
[INFO]     // 1 parameter list
[INFO]     ValDef( // uri: String
[INFO]       <param> <paramaccessor> <triedcooking>
[INFO]       "uri"
[INFO]       <tpt> // tree.tpe=String
[INFO]       <empty>
[INFO]     )
[INFO]     <tpt> // tree.tpe=com.example.core.compute.jobs.factstore.util.FileSystemFactory
[INFO]     Block( // tree.tpe=Unit
[INFO]       // 2 statements
[INFO]       Assign( // tree.tpe=Unit
[INFO]         FileSystemFactory.this."uri" // private[this] val uri: String in class FileSystemFactory, tree.tpe=String
[INFO]         "uri" // uri: String, tree.tpe=String
[INFO]       )
[INFO]       Apply( // def <init>(): Object in class Object, tree.tpe=Object
[INFO]         FileSystemFactory.super."<init>" // def <init>(): Object in class Object, tree.tpe=()Object
[INFO]         Nil
[INFO]       )
[INFO]       ()
[INFO]     )
[INFO]   )
[INFO] )
[INFO]
[INFO] == Expanded type of tree ==
[INFO]
[INFO] ConstantType(
[INFO]   value = Constant(com.example.core.compute.jobs.factstore.source.SourceDirectory)
[INFO]
[INFO] )
[INFO]
[INFO] uncaught exception during compilation: java.io.IOException
[ERROR] error: Operation not permitted
[WARNING] 7 warnings found
[ERROR] two errors found

However when I assign the for-comprehension to a val, it compiles without issue:

val result = for {
     _ <- markerUtil.uploadingStarted(source, destPath, files.map(_.path))
     inputStream <- createInputStream(files)
     _ <- store.upload(destPath, inputStream)
     _ <- markerUtil.uploadingDone(source)
     _ <- deleteFiles(files)
     _ <- markerUtil.uploadedDone(source)
   } yield UploadResult(inputSize, destPath)
   result

I also tried it by converting the for-yield to a flatMap, I get exactly the same errors, and it also gets resolved when I assign it to a val.
@scabug
Copy link
Author

scabug commented Feb 2, 2015

Imported From: https://issues.scala-lang.org/browse/SI-9132?orig=1
Reporter: Nathan Bijnens (nathan_gs)
Affected Versions: 2.10.4

@scabug
Copy link
Author

scabug commented Feb 3, 2015

@retronym said:
Is that SBT output? Can you please run last compile to show the full stack trace of the exception?

To hazard a guess, it sounds like a file system error writing a class file. Perhaps a file name of a class is too long for the underlying filesystem, or perhaps a file is on disk already with permissions that don't allow it to be overwritten.

@scabug
Copy link
Author

scabug commented Feb 3, 2015

@retronym said:
(I've marked this as "incomplete" pending more details, please reopen as soon as you've got them and I'll take a further look)

@scabug
Copy link
Author

scabug commented Feb 3, 2015

Nathan Bijnens (nathan_gs) said:
As indicated in the environment I am using maven 3.

I used this command to make sure I was starting 'fresh':

rm -rf target/ ; mvn clean

Then I tried to compile using following code:

mvn compile -e

Which gave following output:

[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Archive Factstore 1.4.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/maven-metadata.xml
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/maven-metadata.xml
Downloading: http://www.allanbank.com/repo/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/maven-metadata.xml



Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/core-common-datatypes/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/core-common-datatypes/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/core-common-datatypes/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://www.allanbank.com/repo/com/example/core-common-datatypes/1.0.0-SNAPSHOT/maven-metadata.xml



Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: http://www.allanbank.com/repo/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/maven-metadata.xml



Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: http://www.allanbank.com/repo/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml


Downloading: s3://example-artifacts/mvn/snapshot/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
[INFO] Logged in - example-artifacts
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml
607 B   
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml.sha1
        
Downloaded: s3://example-artifacts/mvn/snapshot/com/example/core-compute-util/1.2.0-SNAPSHOT/maven-metadata.xml (607 B at 0.1 KB/sec)
[INFO] Logged off - example-artifacts
[INFO] Transfers: 2 Time: 711ms Amount: 647 bytes Throughput: 909.986 bytes/s
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
        
Downloading: http://www.allanbank.com/repo/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
        
        
        
[INFO] Logged in - example-artifacts
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
788 B   
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml.sha1
        
Downloaded: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml (788 B at 0.4 KB/sec)
[INFO] Logged off - example-artifacts
[INFO] Transfers: 2 Time: 727ms Amount: 828 bytes Throughput: 1.112 KB/s
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
        
Downloading: http://www.allanbank.com/repo/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
        
        
        
[INFO] Logged in - example-artifacts
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
607 B   
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml.sha1
        
Downloaded: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml (607 B at 0.4 KB/sec)
[INFO] Logged off - example-artifacts
[INFO] Transfers: 2 Time: 674ms Amount: 647 bytes Throughput: 959.941 bytes/s
Downloading: https://repository.cloudera.com/artifactory/cloudera-repos/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo1.maven.org/maven2/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
Downloading: https://oss.sonatype.org/content/groups/scala-tools/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
        
Downloading: http://www.allanbank.com/repo/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
        
        
        
[INFO] Logged in - example-artifacts
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml
787 B   
[INFO] Downloading: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml.sha1
        
Downloaded: s3://example-artifacts/mvn/snapshot/com/example/devops-common-cukes-docker/1.4.0-SNAPSHOT/maven-metadata.xml (787 B at 0.4 KB/sec)
[INFO] Logged off - example-artifacts
[INFO] Transfers: 2 Time: 637ms Amount: 827 bytes Throughput: 1.268 KB/s
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:parse-version (parse-version) @ core-compute-jobs-factstore ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.6.5.201403032054:prepare-agent (default-prepare-agent) @ core-compute-jobs-factstore ---
[INFO] argLine set to -javaagent:/home/vagrant/.m2/repository/org/jacoco/org.jacoco.agent/0.6.5.201403032054/org.jacoco.agent-0.6.5.201403032054-runtime.jar=destfile=/vagrant/core-compute-jobs-factstore/target/jacoco.exec
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:add-source (add-scala-sources) @ core-compute-jobs-factstore ---
[INFO] Source directory: /vagrant/core-compute-jobs-factstore/src/main/scala added.
[INFO] 
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ core-compute-jobs-factstore ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ core-compute-jobs-factstore ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-scala-plugin:2.15.2:compile (default) @ core-compute-jobs-factstore ---
[INFO] Checking for multiple versions of scala
Downloading: http://maven.twttr.com/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
Downloading: http://repo.spray.io/com/example/devops-common-cukes-base/1.2.0-SNAPSHOT/maven-metadata.xml
        
        
Downloading: http://repo.spray.io/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
Downloading: http://maven.twttr.com/com/example/devops-common-cukes/1.3.0-SNAPSHOT/maven-metadata.xml
        
        
[WARNING]  Expected all dependencies to require Scala version: 2.10.4
[WARNING]  com.example:core-compute-jobs-factstore:1.4.0-SNAPSHOT requires scala version: 2.10.4
[WARNING]  com.twitter:chill_2.10:0.3.6 requires scala version: 2.10.3
[WARNING] Multiple versions of scala libraries detected!
[INFO] includes = [**/*.scala,**/*.java,]
[INFO] excludes = []
[INFO] /vagrant/core-compute-jobs-factstore/src/main/scala:-1: info: compiling
[INFO] Compiling 14 source files to /vagrant/core-compute-jobs-factstore/target/classes at 1422955019817
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[ERROR] error: 
[INFO]      while compiling: /vagrant/core-compute-jobs-factstore/src/main/scala/com/example/core/compute/jobs/factstore/util/FileSystemFactory.scala
[INFO]         during phase: jvm
[INFO]      library version: version 2.10.4
[INFO]     compiler version: version 2.10.4
[INFO]   reconstructed args: -classpath /vagrant/core-compute-jobs-factstore/target/classes:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-client/2.0.0-mr1-cdh4.4.0/hadoop-client-2.0.0-mr1-cdh4.4.0.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-common/2.0.0-cdh4.4.0/hadoop-common-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-annotations/2.0.0-cdh4.4.0/hadoop-annotations-2.0.0-cdh4.4.0.jar:/usr/lib/jvm/java-7-openjdk-amd64/jre/../lib/tools.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-math/2.1/commons-math-2.1.jar:/home/vagrant/.m2/repository/xmlenc/xmlenc/0.52/xmlenc-0.52.jar:/home/vagrant/.m2/repository/commons-io/commons-io/2.1/commons-io-2.1.jar:/home/vagrant/.m2/repository/commons-el/commons-el/1.0/commons-el-1.0.jar:/home/vagrant/.m2/repository/commons-lang/commons-lang/2.5/commons-lang-2.5.jar:/home/vagrant/.m2/repository/commons-configuration/commons-configuration/1.6/commons-configuration-1.6.jar:/home/vagrant/.m2/repository/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar:/home/vagrant/.m2/repository/commons-digester/commons-digester/1.8/commons-digester-1.8.jar:/home/vagrant/.m2/repository/commons-beanutils/commons-beanutils/1.7.0/commons-beanutils-1.7.0.jar:/home/vagrant/.m2/repository/commons-beanutils/commons-beanutils-core/1.8.0/commons-beanutils-core-1.8.0.jar:/home/vagrant/.m2/repository/org/codehaus/jackson/jackson-core-asl/1.8.8/jackson-core-asl-1.8.8.jar:/home/vagrant/.m2/repository/org/codehaus/jackson/jackson-mapper-asl/1.8.8/jackson-mapper-asl-1.8.8.jar:/home/vagrant/.m2/repository/org/mockito/mockito-all/1.8.5/mockito-all-1.8.5.jar:/home/vagrant/.m2/repository/org/apache/avro/avro/1.7.4/avro-1.7.4.jar:/home/vagrant/.m2/repository/com/thoughtworks/paranamer/paranamer/2.3/paranamer-2.3.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-compress/1.4.1/commons-compress-1.4.1.jar:/home/vagrant/.m2/repository/org/tukaani/xz/1.0/xz-1.0.jar:/home/vagrant/.m2/repository/com/google/protobuf/protobuf-java/2.4.0a/protobuf-java-2.4.0a.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-auth/2.0.0-cdh4.4.0/hadoop-auth-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/com/jcraft/jsch/0.1.42/jsch-0.1.42.jar:/home/vagrant/.m2/repository/org/apache/zookeeper/zookeeper/3.4.5-cdh4.4.0/zookeeper-3.4.5-cdh4.4.0.jar:/home/vagrant/.m2/repository/jline/jline/0.9.94/jline-0.9.94.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-hdfs/2.0.0-cdh4.4.0/hadoop-hdfs-2.0.0-cdh4.4.0.jar:/home/vagrant/.m2/repository/com/sun/jersey/jersey-core/1.8/jersey-core-1.8.jar:/home/vagrant/.m2/repository/com/sun/jersey/jersey-server/1.8/jersey-server-1.8.jar:/home/vagrant/.m2/repository/asm/asm/3.1/asm-3.1.jar:/home/vagrant/.m2/repository/commons-cli/commons-cli/1.2/commons-cli-1.2.jar:/home/vagrant/.m2/repository/org/apache/hadoop/hadoop-core/2.0.0-mr1-cdh4.4.0/hadoop-core-2.0.0-mr1-cdh4.4.0.jar:/home/vagrant/.m2/repository/hsqldb/hsqldb/1.8.0.10/hsqldb-1.8.0.10.jar:/home/vagrant/.m2/repository/com/amazonaws/aws-java-sdk/1.6.4/aws-java-sdk-1.6.4.jar:/home/vagrant/.m2/repository/commons-logging/commons-logging/1.1.1/commons-logging-1.1.1.jar:/home/vagrant/.m2/repository/commons-codec/commons-codec/1.3/commons-codec-1.3.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.1.1/jackson-core-2.1.1.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-databind/2.1.1/jackson-databind-2.1.1.jar:/home/vagrant/.m2/repository/com/fasterxml/jackson/core/jackson-annotations/2.1.1/jackson-annotations-2.1.1.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-library/2.10.4/scala-library-2.10.4.jar:/home/vagrant/.m2/repository/org/apache/spark/spark-core_2.10/1.1.1/spark-core_2.10-1.1.1.jar:/home/vagrant/.m2/repository/net/java/dev/jets3t/jets3t/0.7.1/jets3t-0.7.1.jar:/home/vagrant/.m2/repository/commons-httpclient/commons-httpclient/3.1/commons-httpclient-3.1.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-recipes/2.4.0/curator-recipes-2.4.0.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-framework/2.4.0/curator-framework-2.4.0.jar:/home/vagrant/.m2/repository/org/apache/curator/curator-client/2.4.0/curator-client-2.4.0.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-plus/8.1.14.v20131031/jetty-plus-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.transaction/1.1.1.v201105210645/javax.transaction-1.1.1.v201105210645.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-webapp/8.1.14.v20131031/jetty-webapp-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-xml/8.1.14.v20131031/jetty-xml-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-servlet/8.1.14.v20131031/jetty-servlet-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-jndi/8.1.14.v20131031/jetty-jndi-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.mail.glassfish/1.4.1.v201005082020/javax.mail.glassfish-1.4.1.v201005082020.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.activation/1.1.0.v201105071233/javax.activation-1.1.0.v201105071233.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-security/8.1.14.v20131031/jetty-security-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-util/8.1.14.v20131031/jetty-util-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-server/8.1.14.v20131031/jetty-server-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/orbit/javax.servlet/3.0.0.v201112011016/javax.servlet-3.0.0.v201112011016.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-continuation/8.1.14.v20131031/jetty-continuation-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-http/8.1.14.v20131031/jetty-http-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/org/eclipse/jetty/jetty-io/8.1.14.v20131031/jetty-io-8.1.14.v20131031.jar:/home/vagrant/.m2/repository/com/google/guava/guava/14.0.1/guava-14.0.1.jar:/home/vagrant/.m2/repository/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar:/home/vagrant/.m2/repository/com/google/code/findbugs/jsr305/1.3.9/jsr305-1.3.9.jar:/home/vagrant/.m2/repository/org/slf4j/slf4j-api/1.7.5/slf4j-api-1.7.5.jar:/home/vagrant/.m2/repository/org/slf4j/jul-to-slf4j/1.7.5/jul-to-slf4j-1.7.5.jar:/home/vagrant/.m2/repository/org/slf4j/jcl-over-slf4j/1.7.5/jcl-over-slf4j-1.7.5.jar:/home/vagrant/.m2/repository/log4j/log4j/1.2.17/log4j-1.2.17.jar:/home/vagrant/.m2/repository/org/slf4j/slf4j-log4j12/1.7.5/slf4j-log4j12-1.7.5.jar:/home/vagrant/.m2/repository/com/ning/compress-lzf/1.0.0/compress-lzf-1.0.0.jar:/home/vagrant/.m2/repository/org/xerial/snappy/snappy-java/1.0.5.3/snappy-java-1.0.5.3.jar:/home/vagrant/.m2/repository/net/jpountz/lz4/lz4/1.2.0/lz4-1.2.0.jar:/home/vagrant/.m2/repository/com/twitter/chill_2.10/0.3.6/chill_2.10-0.3.6.jar:/home/vagrant/.m2/repository/com/twitter/chill-java/0.3.6/chill-java-0.3.6.jar:/home/vagrant/.m2/repository/commons-net/commons-net/2.2/commons-net-2.2.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-remote_2.10/2.2.3-shaded-protobuf/akka-remote_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-actor_2.10/2.2.3-shaded-protobuf/akka-actor_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/io/netty/netty/3.6.6.Final/netty-3.6.6.Final.jar:/home/vagrant/.m2/repository/org/spark-project/protobuf/protobuf-java/2.4.1-shaded/protobuf-java-2.4.1-shaded.jar:/home/vagrant/.m2/repository/org/uncommons/maths/uncommons-maths/1.2.2a/uncommons-maths-1.2.2a.jar:/home/vagrant/.m2/repository/org/spark-project/akka/akka-slf4j_2.10/2.2.3-shaded-protobuf/akka-slf4j_2.10-2.2.3-shaded-protobuf.jar:/home/vagrant/.m2/repository/org/json4s/json4s-jackson_2.10/3.2.10/json4s-jackson_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/json4s/json4s-core_2.10/3.2.10/json4s-core_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/json4s/json4s-ast_2.10/3.2.10/json4s-ast_2.10-3.2.10.jar:/home/vagrant/.m2/repository/org/scala-lang/scalap/2.10.0/scalap-2.10.0.jar:/home/vagrant/.m2/repository/colt/colt/1.2.0/colt-1.2.0.jar:/home/vagrant/.m2/repository/concurrent/concurrent/1.3.4/concurrent-1.3.4.jar:/home/vagrant/.m2/repository/org/apache/mesos/mesos/0.18.1/mesos-0.18.1-shaded-protobuf.jar:/home/vagrant/.m2/repository/io/netty/netty-all/4.0.23.Final/netty-all-4.0.23.Final.jar:/home/vagrant/.m2/repository/com/clearspring/analytics/stream/2.7.0/stream-2.7.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-core/3.0.0/metrics-core-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-jvm/3.0.0/metrics-jvm-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-json/3.0.0/metrics-json-3.0.0.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-graphite/3.0.0/metrics-graphite-3.0.0.jar:/home/vagrant/.m2/repository/org/tachyonproject/tachyon-client/0.5.0/tachyon-client-0.5.0.jar:/home/vagrant/.m2/repository/org/tachyonproject/tachyon/0.5.0/tachyon-0.5.0.jar:/home/vagrant/.m2/repository/org/spark-project/pyrolite/2.0.1/pyrolite-2.0.1.jar:/home/vagrant/.m2/repository/net/sf/py4j/py4j/0.8.2.1/py4j-0.8.2.1.jar:/home/vagrant/.m2/repository/joda-time/joda-time/2.2/joda-time-2.2.jar:/home/vagrant/.m2/repository/org/apache/httpcomponents/httpclient/4.2.6/httpclient-4.2.6.jar:/home/vagrant/.m2/repository/org/apache/httpcomponents/httpcore/4.2.5/httpcore-4.2.5.jar:/home/vagrant/.m2/repository/com/esotericsoftware/kryo/kryo/2.22/kryo-2.22.jar:/home/vagrant/.m2/repository/com/typesafe/config/1.2.1/config-1.2.1.jar:/home/vagrant/.m2/repository/com/example/core-common-datatypes-domain/1.6.0-SNAPSHOT/core-common-datatypes-domain-1.6.0-SNAPSHOT.jar:/home/vagrant/.m2/repository/org/apache/thrift/libthrift/0.9.1/libthrift-0.9.1.jar:/home/vagrant/.m2/repository/com/twitter/scrooge-core_2.10/3.12.3/scrooge-core_2.10-3.12.3.jar:/home/vagrant/.m2/repository/com/example/core-compute-util-spark-metrics/1.0.0-SNAPSHOT/core-compute-util-spark-metrics-1.0.0-SNAPSHOT.jar:/home/vagrant/.m2/repository/com/codahale/metrics/metrics-ganglia/3.0.1/metrics-ganglia-3.0.1.jar:/home/vagrant/.m2/repository/info/ganglia/gmetric4j/gmetric4j/1.0.3/gmetric4j-1.0.3.jar:/home/vagrant/.m2/repository/org/acplt/oncrpc/1.0.7/oncrpc-1.0.7.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-reflect/2.10.3/scala-reflect-2.10.3.jar:/home/vagrant/.m2/repository/junit/junit/4.11/junit-4.11.jar:/home/vagrant/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar:/home/vagrant/.m2/repository/org/scala-lang/scala-compiler/2.10.3/scala-compiler-2.10.3.jar -d /vagrant/core-compute-jobs-factstore/target/classes
[INFO] 
[INFO]   last tree to typer: Literal(Constant(com.example.core.compute.jobs.factstore.source.SourceDirectory))
[INFO]               symbol: null
[INFO]    symbol definition: null
[INFO]                  tpe: Class(classOf[com.example.core.compute.jobs.factstore.source.SourceDirectory])
[INFO]        symbol owners: 
[INFO]       context owners: class FileSystemFactory -> package util
[INFO] 
[INFO] == Enclosing template or block ==
[INFO] 
[INFO] Template( // val <local FileSystemFactory>: <notype> in class FileSystemFactory, tree.tpe=com.example.core.compute.jobs.factstore.util.FileSystemFactory
[INFO]   "java.lang.Object", "scala.Serializable" // parents
[INFO]   ValDef(
[INFO]     private
[INFO]     "_"
[INFO]     <tpt>
[INFO]     <empty>
[INFO]   )
[INFO]   // 3 statements
[INFO]   ValDef( // private[this] val uri: String in class FileSystemFactory
[INFO]     private <local> <paramaccessor> <triedcooking>
[INFO]     "uri"
[INFO]     <tpt> // tree.tpe=String
[INFO]     <empty>
[INFO]   )
[INFO]   DefDef( // def get(): org.apache.hadoop.fs.FileSystem in class FileSystemFactory
[INFO]     <method> <triedcooking>
[INFO]     "get"
[INFO]     []
[INFO]     List(Nil)
[INFO]     <tpt> // tree.tpe=org.apache.hadoop.fs.FileSystem
[INFO]     Apply( // def get(x$1: java.net.URI,x$2: org.apache.hadoop.conf.Configuration): org.apache.hadoop.fs.FileSystem in object FileSystem, tree.tpe=org.apache.hadoop.fs.FileSystem
[INFO]       "org"."apache"."hadoop"."fs"."HFileSystem"."get" // def get(x$1: java.net.URI,x$2: org.apache.hadoop.conf.Configuration): org.apache.hadoop.fs.FileSystem in object FileSystem, tree.tpe=(x$1: java.net.URI, x$2: org.apache.hadoop.conf.Configuration)org.apache.hadoop.fs.FileSystem
[INFO]       // 2 arguments
[INFO]       Apply( // def <init>(x$1: String): java.net.URI in class URI, tree.tpe=java.net.URI
[INFO]         new java.net.URI."<init>" // def <init>(x$1: String): java.net.URI in class URI, tree.tpe=(x$1: String)java.net.URI
[INFO]         FileSystemFactory.this."uri" // private[this] val uri: String in class FileSystemFactory, tree.tpe=String
[INFO]       )
[INFO]       Apply( // def <init>(): org.apache.hadoop.conf.Configuration in class Configuration, tree.tpe=org.apache.hadoop.conf.Configuration
[INFO]         new org.apache.hadoop.conf.Configuration."<init>" // def <init>(): org.apache.hadoop.conf.Configuration in class Configuration, tree.tpe=()org.apache.hadoop.conf.Configuration
[INFO]         Nil
[INFO]       )
[INFO]     )
[INFO]   )
[INFO]   DefDef( // def <init>(uri: String): com.example.core.compute.jobs.factstore.util.FileSystemFactory in class FileSystemFactory
[INFO]     <method> <triedcooking>
[INFO]     "<init>"
[INFO]     []
[INFO]     // 1 parameter list
[INFO]     ValDef( // uri: String
[INFO]       <param> <paramaccessor> <triedcooking>
[INFO]       "uri"
[INFO]       <tpt> // tree.tpe=String
[INFO]       <empty>
[INFO]     )
[INFO]     <tpt> // tree.tpe=com.example.core.compute.jobs.factstore.util.FileSystemFactory
[INFO]     Block( // tree.tpe=Unit
[INFO]       // 2 statements
[INFO]       Assign( // tree.tpe=Unit
[INFO]         FileSystemFactory.this."uri" // private[this] val uri: String in class FileSystemFactory, tree.tpe=String
[INFO]         "uri" // uri: String, tree.tpe=String
[INFO]       )
[INFO]       Apply( // def <init>(): Object in class Object, tree.tpe=Object
[INFO]         FileSystemFactory.super."<init>" // def <init>(): Object in class Object, tree.tpe=()Object
[INFO]         Nil
[INFO]       )
[INFO]       ()
[INFO]     )
[INFO]   )
[INFO] )
[INFO] 
[INFO] == Expanded type of tree ==
[INFO] 
[INFO] ConstantType(
[INFO]   value = Constant(com.example.core.compute.jobs.factstore.source.SourceDirectory)
[INFO]   
[INFO] )
[INFO] 
[INFO] uncaught exception during compilation: java.io.IOException
[ERROR] error: Operation not permitted
[WARNING] 7 warnings found
[ERROR] two errors found
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:11.629s
[INFO] Finished at: Tue Feb 03 09:18:55 UTC 2015
[INFO] Final Memory: 20M/50M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project core-compute-jobs-factstore: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1) -> [Help 1]
org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.scala-tools:maven-scala-plugin:2.15.2:compile (default) on project core-compute-jobs-factstore: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:217)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84)
	at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:59)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.singleThreadedBuild(LifecycleStarter.java:183)
	at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:161)
	at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:320)
	at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:156)
	at org.apache.maven.cli.MavenCli.execute(MavenCli.java:537)
	at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:196)
	at org.apache.maven.cli.MavenCli.main(MavenCli.java:141)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
	at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
	at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
	at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)
Caused by: org.apache.maven.plugin.MojoExecutionException: wrap: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
	at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:350)
	at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101)
	at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209)
	... 19 more
Caused by: org.apache.commons.exec.ExecuteException: Process exited with an error: 1(Exit value: 1)
	at org.apache.commons.exec.DefaultExecutor.executeInternal(DefaultExecutor.java:346)
	at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:149)
	at org.apache.commons.exec.DefaultExecutor.execute(DefaultExecutor.java:136)
	at org_scala_tools_maven_executions.JavaMainCallerByFork.run(JavaMainCallerByFork.java:80)
	at org_scala_tools_maven.ScalaCompilerSupport.compile(ScalaCompilerSupport.java:124)
	at org_scala_tools_maven.ScalaCompilerSupport.doExecute(ScalaCompilerSupport.java:80)
	at org_scala_tools_maven.ScalaMojoSupport.execute(ScalaMojoSupport.java:342)
	... 21 more
[ERROR] 
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

If I assign the for-comprehension to a value, repeat all the steps (rm target & clean, mvn compile) it compiles without error. I am very sure it's not a permission or disk issue.

[INFO] Error stacktraces are turned on.
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Archive Factstore 1.4.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:parse-version (parse-version) @ core-compute-jobs-factstore ---
[INFO] 
[INFO] --- jacoco-maven-plugin:0.6.5.201403032054:prepare-agent (default-prepare-agent) @ core-compute-jobs-factstore ---
[INFO] argLine set to -javaagent:/home/vagrant/.m2/repository/org/jacoco/org.jacoco.agent/0.6.5.201403032054/org.jacoco.agent-0.6.5.201403032054-runtime.jar=destfile=/vagrant/core-compute-jobs-factstore/target/jacoco.exec
[INFO] 
[INFO] --- build-helper-maven-plugin:1.8:add-source (add-scala-sources) @ core-compute-jobs-factstore ---
[INFO] Source directory: /vagrant/core-compute-jobs-factstore/src/main/scala added.
[INFO] 
[INFO] --- maven-resources-plugin:2.3:resources (default-resources) @ core-compute-jobs-factstore ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ core-compute-jobs-factstore ---
[INFO] Nothing to compile - all classes are up to date
[INFO] 
[INFO] --- maven-scala-plugin:2.15.2:compile (default) @ core-compute-jobs-factstore ---
[INFO] Checking for multiple versions of scala
[WARNING]  Expected all dependencies to require Scala version: 2.10.4
[WARNING]  com.example:core-compute-jobs-factstore:1.4.0-SNAPSHOT requires scala version: 2.10.4
[WARNING]  com.twitter:chill_2.10:0.3.6 requires scala version: 2.10.3
[WARNING] Multiple versions of scala libraries detected!
[INFO] includes = [**/*.scala,**/*.java,]
[INFO] excludes = []
[INFO] /vagrant/core-compute-jobs-factstore/src/main/scala:-1: info: compiling
[INFO] Compiling 14 source files to /vagrant/core-compute-jobs-factstore/target/classes at 1422955813091
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.FromString not found - continuing with a stub.
[WARNING] warning: Class org.joda.convert.ToString not found - continuing with a stub.
[WARNING] warning: there were 2 deprecation warning(s); re-run with -deprecation for details
[WARNING] 8 warnings found
[INFO] prepare-compile in 0 s
[INFO] compile in 148 s
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 4:18.537s
[INFO] Finished at: Tue Feb 03 09:32:41 UTC 2015
[INFO] Final Memory: 16M/39M
[INFO] ------------------------------------------------------------------------

@scabug
Copy link
Author

scabug commented Feb 3, 2015

@retronym said:
Sorry, I didn't spot that you'd already specified Mavan in the bug report. Unfortunately it seems to be swallowing the interesting part of the exception stack trace (in a similar manner to the default behaviour of SBT!)

Is your project in an encrypted home directory in Ubuntu? The long class names generated by heavily nested closures (as are generated in for comprehensions with many generators) can exceed the maximum file path allowed. See for example scala/pickling#10 / http://brianoneill.blogspot.com.au/2012/10/file-name-too-long-in-scalasbt-on-ubuntu.html

@scabug
Copy link
Author

scabug commented Feb 3, 2015

Nathan Bijnens (nathan_gs) said:
I am not using any encryption. I don't think it has something to do with the length of the file paths.

I find it very strange that this doesn't work:

for {
     _ <- markerUtil.uploadingStarted(source, destPath, files.map(_.path)) // Try[Unit]
     inputStream <- createInputStream(files) // Try[InputStream]
     _ <- store.upload(destPath, inputStream) // Try[Unit]
     _ <- markerUtil.uploadingDone(source) // Try[Unit]
     _ <- deleteFiles(files) // Try[Unit]
     _ <- markerUtil.uploadedDone(source) // Try[Unit]
   } yield UploadResult(inputSize, destPath)

while this works:

val result = for {
     _ <- markerUtil.uploadingStarted(source, destPath, files.map(_.path))
     inputStream <- createInputStream(files)
     _ <- store.upload(destPath, inputStream)
     _ <- markerUtil.uploadingDone(source)
     _ <- deleteFiles(files)
     _ <- markerUtil.uploadedDone(source)
   } yield UploadResult(inputSize, destPath)
result

@scabug
Copy link
Author

scabug commented Feb 3, 2015

@retronym said (edited on Feb 3, 2015 12:50:59 PM UTC):
The two snippets subtly change the way that anonymous classes are named; they end up including the method name as a prefix of the mangled name in one case but not in the other.

% cat sandbox/test.scala
object Test {
  def test = {
    val result = for {
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
     } yield ()
   }
}

% scalac-hash v2.11.5 -d /tmp/classes sandbox/test.scala

% find /tmp/classes
/tmp/classes/
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3$$anonfun$apply$4$$anonfun$apply$5.class
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3$$anonfun$apply$4.class
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3.class
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1$$anonfun$apply$2.class
/tmp/classes/Test$$anonfun$1$$anonfun$apply$1.class
/tmp/classes/Test$$anonfun$1.class
/tmp/classes/Test$.class
/tmp/classes/Test$markerUtil$.class
/tmp/classes/Test$store$.class
/tmp/classes/Test$UploadResult$.class
/tmp/classes/Test$UploadResult.class
/tmp/classes/Test.class

% rm -rf /tmp/classes/*

% subl sandbox/test.scala

% cat sandbox/test.scala
object Test {
  def test = {
    for {
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
     } yield ()
   }
}

% scalac-hash v2.11.5 -d /tmp/classes sandbox/test.scala

% find /tmp/classes
/tmp/classes
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3$$anonfun$apply$4$$anonfun$apply$5.class
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3$$anonfun$apply$4.class
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1$$anonfun$apply$2$$anonfun$apply$3.class
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1$$anonfun$apply$2.class
/tmp/classes/Test$$anonfun$test$1$$anonfun$apply$1.class
/tmp/classes/Test$$anonfun$test$1.class
/tmp/classes/Test$.class
/tmp/classes/Test$markerUtil$.class
/tmp/classes/Test$store$.class
/tmp/classes/Test$UploadResult$.class
/tmp/classes/Test$UploadResult.class
/tmp/classes/Test.class

Could I ask you to try to compile this program on your system using the command line version of the Scala compiler?

object Test {
  def test = {
    val r = for {
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
       _ <- Nil
     } yield ()
   }
}

@scabug
Copy link
Author

scabug commented Feb 3, 2015

@retronym said:
Could you also please double check for encrypted home partition with these instructions? http://askubuntu.com/questions/53242/check-if-partition-is-encrypted

@scabug
Copy link
Author

scabug commented Feb 11, 2015

Nathan Bijnens (nathan_gs) said:
It was on a Vagrant/Virtualbox machine, with a Windows host, apparently there is a path length limit, which gives strange errors like these. This issue can be closed, although the error messages could be a little more cleaner (for instance, a reference to the path length).

@som-snytt
Copy link

@retronym This was a nice one to read.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@retronym retronym

Labels
compiler crash
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@retronym @SethTisue @som-snytt @scabug