New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Local Spark jobs no longer work with `sbt run` #3647

Closed
fosskers opened this Issue Oct 18, 2017 · 8 comments

Comments

Projects
None yet
5 participants
@fosskers

fosskers commented Oct 18, 2017

I updated a few of my projects to sbt 1.0.2 and found this unexpected issue. I don't know enough about the architecture of the 1.X series of sbt, so I'm not sure what could be causing the problem.

steps

Try sbt run on the following minimal example:

package com.example

import org.apache.spark._
import org.apache.spark.rdd._

// --- //

object Hello {
  def main(args: Array[String]): Unit = {
    val conf = new SparkConf()
      .setIfMissing("spark.master", "local[*]")
      .setAppName("hello-spark")

    implicit val sc: SparkContext = new SparkContext(conf)

    val nums: RDD[Int] = sc.parallelize(List(1,2,3))

    println(nums.count)

    sc.stop()
  }
}

build.sbt:

name := """spark-helloworld"""

version := "1.0"

scalaVersion := "2.11.11"

libraryDependencies ++= Seq(
  "org.apache.spark" %% "spark-hive" % "2.2.0"
)

problem

With sbt 1.0.2, we get the runtime error:

17/10/18 10:29:41 ERROR Utils: Exception encountered
java.lang.ClassNotFoundException: scala.collection.mutable.WrappedArray$ofInt
	at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	at java.lang.Class.forName0(Native Method)
	at java.lang.Class.forName(Class.java:348)
...

The real-world case where I saw this had a bit of a different failure, but they both seems to be class path issues:

[error] java.lang.LinkageError: loader constraint violation in interface itable initialization: when resolving method "org.apache.spark.sql.hive.client.HiveClientImpl.listDatabases(Ljava/lang/String;)Lscala/collection/Seq;" the class loader (instance of org/apache/spark/sql/hive/client/IsolatedClientLoader$$anon$1) of the current class, org/apache/spark/sql/hive/client/HiveClientImpl, and the class loader (instance of sbt/internal/inc/classpath/ClasspathUtilities$$anon$1) for interface org/apache/spark/sql/hive/client/HiveClient have different Class objects for the type scala/collection/Seq used in the signature
[error]     at java.lang.Class.getDeclaredConstructors0(Native Method)
[error]     at java.lang.Class.privateGetDeclaredConstructors(Class.java:2671)
[error]     at java.lang.Class.getConstructors(Class.java:1651)

expectation

We should see the number 3 printed, as it does with sbt 0.13.6. Being able to do sbt run on tiny Spark jobs is convenient, since it's faster than forming a proper spark-submit.

notes

sbt version: 1.0.2 (works fine with 0.13.16).

@xerial

This comment has been minimized.

Show comment
Hide comment
@xerial

xerial Oct 20, 2017

Contributor

I think classpath issue of run command is related #3608

Contributor

xerial commented Oct 20, 2017

I think classpath issue of run command is related #3608

@fosskers

This comment has been minimized.

Show comment
Hide comment
@fosskers

fosskers Oct 20, 2017

fosskers commented Oct 20, 2017

@slim-azaiz

This comment has been minimized.

Show comment
Hide comment
@slim-azaiz

slim-azaiz Nov 24, 2017

it does not work @fosskers

slim-azaiz commented Nov 24, 2017

it does not work @fosskers

@fosskers

This comment has been minimized.

Show comment
Hide comment
@fosskers

fosskers Nov 24, 2017

It does for me with fork in run := true in my build.sbt.

fosskers commented Nov 24, 2017

It does for me with fork in run := true in my build.sbt.

@slim-azaiz

This comment has been minimized.

Show comment
Hide comment
@slim-azaiz

slim-azaiz Nov 27, 2017

@fosskers yes it works but i got these errors : root = sun.misc.Launcher$AppClassLoader@677327b6 cp = Set(/tmp/sbt_7d2cae42/target/a10732c7/protobuf-java-2.5.0.jar, /tmp/sbt_7d2cae42/target/7a9adf97/jersey-media-jaxb-2.22.2.jar, /tmp/sbt_7d2cae42/target/23807d9f/hadoop-yarn-server-common-2.6.5.jar, /tmp/sbt_7d2cae42/target/4ed2b2d4/osgi-resource-locator-1.0.1.jar, /tmp/sbt_7d2cae42/target/b67c76af/scala-xml_2.11-1.0.4.jar, /tmp/sbt_7d2cae42/target/ed364e77/metrics-jvm-3.1.2.jar, /tmp/sbt_7d2cae42/target/35a3c69e/guava-11.0.2.jar, /tmp/sbt_7d2cae42/target/3940343e/hbase-protocol-1.0.3.jar, /tmp/sbt_7d2cae42/target/2973d150/junit-4.12.jar, /tmp/sbt_7d2cae42/target/07ce5e98/cglib-2.2.1-v20090111.jar, /tmp/sbt_7d2cae42/target/65fbf0c7/jets3t-0.9.3.jar, /tmp/sbt_7d2cae42/target/9781746a/netty-all-4.0.43.Final.jar, /tmp/sbt_7d2cae42/target/6925c778/disruptor-3.3.0.jar, /tmp/sbt_7d2cae42/target/5bd604c1/parquet-column-1.8.2.jar, /tmp/sbt_7d2cae42/target/f37255e3/hbase-server-1.0.3.jar, /tmp/sbt_7d2cae42/target/534d72d2/jackson-jaxrs-1.9.13.jar, /tmp/sbt_7d2cae42/target/49f4a68d/hbase-prefix-tree-1.0.3.jar, /tmp/sbt_7d2cae42/target/8dbece93/py4j-0.10.4.jar, /tmp/sbt_7d2cae42/target/a6b11447/findbugs-annotations-1.3.9-1.jar, /tmp/sbt_7d2cae42/target/75a8bd7c/jackson-module-scala_2.11-2.8.7.jar, /tmp/sbt_7d2cae42/target/3cbb2bf5/chill_2.11-0.8.0.jar, /tmp/sbt_7d2cae42/target/104e9c2b/javax.ws.rs-api-2.0.1.jar, /tmp/sbt_7d2cae42/target/964cd741/commons-httpclient-3.1.jar, /tmp/sbt_7d2cae42/target/4d4dde44/spark-catalyst_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/485de3a2/activation-1.1.1.jar, /tmp/sbt_7d2cae42/target/07993c12/commons-net-2.2.jar, /tmp/sbt_7d2cae42/target/9ca7d55d/netty-3.10.5.Final.jar, /tmp/sbt_7d2cae42/target/a510bccf/spark-sketch_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/619eba74/paranamer-2.8.jar, /tmp/sbt_7d2cae42/target/1c16e4e4/apacheds-kerberos-codec-2.0.0-M15.jar, /tmp/sbt_7d2cae42/target/479c1e06/javax.annotation-api-1.2.jar, /tmp/sbt_7d2cae42/target/07407d69/jackson-annotations-2.8.7.jar, /tmp/sbt_7d2cae42/target/9ab8afcc/bcprov-jdk15on-1.51.jar, /tmp/sbt_7d2cae42/target/8906a685/common-config-3.3.1.jar, /tmp/sbt_7d2cae42/target/1df2c042/commons-el-1.0.jar, /tmp/sbt_7d2cae42/target/139535a6/slf4j-api-1.7.21.jar, /tmp/sbt_7d2cae42/target/d8f23aff/avro-mapred-1.7.7-hadoop2.jar, /tmp/sbt_7d2cae42/target/fe1285c9/scala-compiler-2.11.8.jar, /tmp/sbt_7d2cae42/target/ae23df9c/jamon-runtime-2.3.1.jar, /tmp/sbt_7d2cae42/target/6975da39/javax.inject-1.jar, /tmp/sbt_7d2cae42/target/6b7c052b/spark-unsafe_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/8341846f/jersey-core-1.9.jar, /tmp/sbt_7d2cae42/target/a90a8719/kafka-schema-registry-client-3.3.1.jar, /tmp/sbt_7d2cae42/target/7ab1c85a/chill-java-0.8.0.jar, /tmp/sbt_7d2cae42/target/0ef148a1/janino-3.0.0.jar, /tmp/sbt_7d2cae42/target/f91b7a4a/httpcore-4.3.3.jar, /tmp/sbt_7d2cae42/target/ff07b5f1/minlog-1.3.0.jar, /tmp/sbt_7d2cae42/target/88d9e57e/metrics-json-3.1.2.jar, /tmp/sbt_7d2cae42/target/23101cc7/json4s-jackson_2.11-3.2.11.jar, /tmp/sbt_7d2cae42/target/f82c035c/metrics-core-2.2.0.jar, /tmp/sbt_7d2cae42/target/8bd0f95e/hadoop-hdfs-2.6.5.jar, /tmp/sbt_7d2cae42/target/4a6507aa/java-xmlbuilder-1.0.jar, /tmp/sbt_7d2cae42/target/5ede3e5f/jersey-server-2.22.2.jar, /tmp/sbt_7d2cae42/job-1/target/403713c1/aziz_2.11-0.1.jar, /tmp/sbt_7d2cae42/target/ddd5a8bc/scala-library-2.11.8.jar, /tmp/sbt_7d2cae42/target/7f567c6b/hadoop-yarn-api-2.6.5.jar, /tmp/sbt_7d2cae42/target/4bfc12ad/commons-logging-1.2.jar, /tmp/sbt_7d2cae42/target/5e6486ff/api-asn1-api-1.0.0-M20.jar, /tmp/sbt_7d2cae42/target/9add0585/mail-1.4.7.jar, /tmp/sbt_7d2cae42/target/96ca5528/jasper-runtime-5.5.23.jar, /tmp/sbt_7d2cae42/target/a60a5e99/gson-2.2.4.jar, /tmp/sbt_7d2cae42/target/22351b44/hadoop-mapreduce-client-common-2.6.5.jar, /tmp/sbt_7d2cae42/target/7d2e2cea/base64-2.3.8.jar, /tmp/sbt_7d2cae42/target/32cadde2/commons-configuration-1.6.jar, /tmp/sbt_7d2cae42/target/dc366f22/hadoop-auth-2.6.5.jar, /tmp/sbt_7d2cae42/target/7422c693/jersey-guava-2.22.2.jar, /tmp/sbt_7d2cae42/target/87d4c44d/json4s-core_2.11-3.2.11.jar, /tmp/sbt_7d2cae42/target/48d92871/snappy-java-1.1.2.6.jar, /tmp/sbt_7d2cae42/target/3e1495b0/compress-lzf-1.0.3.jar, /tmp/sbt_7d2cae42/target/47bf147f/mx4j-3.0.2.jar, /tmp/sbt_7d2cae42/target/9d84f15f/guice-3.0.jar, /tmp/sbt_7d2cae42/target/e6b04760/RoaringBitmap-0.5.11.jar, /tmp/sbt_7d2cae42/target/dc6a73fd/commons-digester-1.8.jar, /tmp/sbt_7d2cae42/target/71c61c84/apacheds-i18n-2.0.0-M15.jar, /tmp/sbt_7d2cae42/target/8736b0fc/curator-recipes-2.6.0.jar, /tmp/sbt_7d2cae42/target/d90487b9/jersey-container-servlet-2.22.2.jar, /tmp/sbt_7d2cae42/target/5abe4c24/ivy-2.4.0.jar, /tmp/sbt_7d2cae42/target/034e4807/jcl-over-slf4j-1.7.16.jar, /tmp/sbt_7d2cae42/target/91e3146d/avro-1.8.2.jar, /tmp/sbt_7d2cae42/target/707350a2/leveldbjni-all-1.8.jar, /tmp/sbt_7d2cae42/target/d6337b0d/stax-api-1.0-2.jar, /tmp/sbt_7d2cae42/target/4b95f489/commons-codec-1.10.jar, /tmp/sbt_7d2cae42/target/0235ba8b/aopalliance-1.0.jar, /tmp/sbt_7d2cae42/target/7938f66b/commons-crypto-1.0.0.jar, /tmp/sbt_7d2cae42/target/1a01a2a1/jettison-1.1.jar, /tmp/sbt_7d2cae42/target/6c6c702c/commons-lang3-3.5.jar, /tmp/sbt_7d2cae42/target/7238b064/slf4j-log4j12-1.7.21.jar, /tmp/sbt_7d2cae42/target/8ad72fe3/commons-collections-3.2.2.jar, /tmp/sbt_7d2cae42/target/f1a3bc53/parquet-jackson-1.8.2.jar, /tmp/sbt_7d2cae42/target/5d4c38ae/kafka_2.11-0.8.2.1.jar, /tmp/sbt_7d2cae42/target/3d5e856d/aopalliance-repackaged-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/8e73d3de/hbase-annotations-1.0.3.jar, /tmp/sbt_7d2cae42/target/c157def1/asm-3.1.jar, /tmp/sbt_7d2cae42/target/77b2fb4c/spark-avro_2.11-4.0.0.jar, /tmp/sbt_7d2cae42/target/4c47155e/httpclient-4.3.6.jar, /tmp/sbt_7d2cae42/target/8613ae82/validation-api-1.1.0.Final.jar, /tmp/sbt_7d2cae42/target/2008b781/hadoop-mapreduce-client-shuffle-2.6.5.jar, /tmp/sbt_7d2cae42/target/9ef74113/kafka-clients-0.8.2.1.jar, /tmp/sbt_7d2cae42/target/4b81d62b/spark-network-shuffle_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/a6a3d493/javax.inject-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/29262f09/hadoop-annotations-2.6.5.jar, /tmp/sbt_7d2cae42/target/1f08024e/joni-2.1.2.jar, /tmp/sbt_7d2cae42/target/1209b898/jersey-common-2.22.2.jar, /tmp/sbt_7d2cae42/target/0e1222db/scalap-2.11.8.jar, /tmp/sbt_7d2cae42/target/b71a4668/pyrolite-4.13.jar, /tmp/sbt_7d2cae42/target/370b5cff/common-utils-3.3.1.jar, /tmp/sbt_7d2cae42/target/c56c5d4e/spark-sql_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/f04ea614/hadoop-yarn-client-2.6.5.jar, /tmp/sbt_7d2cae42/target/c3210db2/spark-streaming-kafka-0-8_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/fd071844/univocity-parsers-2.2.1.jar, /tmp/sbt_7d2cae42/target/f73606e7/htrace-core-3.1.0-incubating.jar, /tmp/sbt_7d2cae42/target/211fe2e6/parquet-hadoop-1.8.2.jar, /tmp/sbt_7d2cae42/target/7369d653/scala-parser-combinators_2.11-1.0.4.jar, /tmp/sbt_7d2cae42/target/205fe37a/unused-1.0.0.jar, /tmp/sbt_7d2cae42/target/b1b6ea3b/commons-io-2.4.jar, /tmp/sbt_7d2cae42/target/0988e998/hadoop-mapreduce-client-jobclient-2.6.5.jar, /tmp/sbt_7d2cae42/target/aeb3021c/jaxb-api-2.2.2.jar, /tmp/sbt_7d2cae42/target/756b3fac/jsp-api-2.1-6.1.14.jar, /tmp/sbt_7d2cae42/target/4877b85d/commons-math-2.2.jar, /tmp/sbt_7d2cae42/target/38383ecc/hadoop-mapreduce-client-core-2.6.5.jar, /tmp/sbt_7d2cae42/target/977d6b42/kryo-shaded-3.0.3.jar, /tmp/sbt_7d2cae42/target/2d5b546c/jul-to-slf4j-1.7.16.jar, /tmp/sbt_7d2cae42/target/3ac44a86/commons-math3-3.4.1.jar, /tmp/sbt_7d2cae42/target/1712fff0/jersey-client-2.22.2.jar, /tmp/sbt_7d2cae42/target/1017432e/hk2-api-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/50d7a8e2/hbase-client-1.0.3.jar, /tmp/sbt_7d2cae42/target/1c36c81e/jackson-databind-2.8.4.jar, /tmp/sbt_7d2cae42/target/d9a09f77/javassist-3.18.1-GA.jar, /tmp/sbt_7d2cae42/target/c54d4b5a/zkclient-0.10.jar, /tmp/sbt_7d2cae42/target/9998f8cf/stream-2.7.0.jar, /tmp/sbt_7d2cae42/target/a871abf0/api-util-1.0.0-M20.jar, /tmp/sbt_7d2cae42/target/3c304d70/jackson-core-asl-1.9.13.jar, /tmp/sbt_7d2cae42/target/523114bb/kafka-avro-serializer-3.3.1.jar, /tmp/sbt_7d2cae42/target/542df0ff/spark-tags_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/3a6ea7cc/jersey-server-1.9.jar, /tmp/sbt_7d2cae42/target/90b215f4/xml-apis-1.3.04.jar, /tmp/sbt_7d2cae42/target/60367999/jetty-sslengine-6.1.26.jar, /tmp/sbt_7d2cae42/target/c708bb25/lz4-1.3.0.jar, /tmp/sbt_7d2cae42/target/7bc7e49d/xercesImpl-2.9.1.jar, /tmp/sbt_7d2cae42/target/d82554ef/xmlenc-0.52.jar, /tmp/sbt_7d2cae42/target/08eebdbb/zookeeper-3.4.10.jar, /tmp/sbt_7d2cae42/target/aacce184/hk2-utils-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/33fa45fd/jcodings-1.0.8.jar, /tmp/sbt_7d2cae42/target/b9007c35/curator-client-2.6.0.jar, /tmp/sbt_7d2cae42/target/81a699c3/curator-framework-2.6.0.jar, /tmp/sbt_7d2cae42/target/c318b7a4/hadoop-yarn-common-2.6.5.jar, /tmp/sbt_7d2cae42/target/f1e6dbc3/spark-core_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/3cd63d07/javax.servlet-api-3.1.0.jar, /tmp/sbt_7d2cae42/target/1451fc3e/hk2-locator-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/f317ae40/spark-launcher_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/336cc5c7/spark-streaming_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/99a18e9a/jline-0.9.94.jar, /tmp/sbt_7d2cae42/target/1ee2f2be/jackson-mapper-asl-1.9.13.jar, /tmp/sbt_7d2cae42/target/775c6ba9/hadoop-common-2.6.5.jar, /tmp/sbt_7d2cae42/target/42a25dc3/hamcrest-core-1.3.jar, /tmp/sbt_7d2cae42/target/5675fd96/commons-beanutils-1.7.0.jar, /tmp/sbt_7d2cae42/target/212c534c/jersey-container-servlet-core-2.22.2.jar, /tmp/sbt_7d2cae42/target/8b46f39c/jackson-core-2.8.7.jar, /tmp/sbt_7d2cae42/target/2bf96b7a/commons-cli-1.2.jar, /tmp/sbt_7d2cae42/target/d7c75374/hadoop-client-2.6.5.jar, /tmp/sbt_7d2cae42/target/2609e36f/antlr4-runtime-4.5.3.jar, /tmp/sbt_7d2cae42/target/0ce1edb9/commons-lang-2.6.jar, /tmp/sbt_7d2cae42/target/15a68399/metrics-graphite-3.1.2.jar, /tmp/sbt_7d2cae42/target/2a3e3137/parquet-format-2.3.1.jar, /tmp/sbt_7d2cae42/target/175dc721/commons-beanutils-core-1.8.0.jar, /tmp/sbt_7d2cae42/target/b754e613/avro-ipc-1.7.7.jar, /tmp/sbt_7d2cae42/target/765ab4bb/parquet-encoding-1.8.2.jar, /tmp/sbt_7d2cae42/target/3a1df1e8/jsp-2.1-6.1.14.jar, /tmp/sbt_7d2cae42/target/1ef0eeef/avro-ipc-1.7.7-tests.jar, /tmp/sbt_7d2cae42/target/6d35bd7f/jasper-compiler-5.5.23.jar, /tmp/sbt_7d2cae42/target/224f03af/metrics-core-3.1.2.jar, /tmp/sbt_7d2cae42/target/a413bb5a/xbean-asm5-shaded-4.4.jar, /tmp/sbt_7d2cae42/target/e3480072/jackson-xc-1.9.13.jar, /tmp/sbt_7d2cae42/target/5af35056/log4j-1.2.17.jar, /tmp/sbt_7d2cae42/target/fd7cc70f/parquet-common-1.8.2.jar, /tmp/sbt_7d2cae42/target/d7461828/htrace-core-3.0.4.jar, /tmp/sbt_7d2cae42/target/a2fd5fbc/spark-hbase-connector_2.10-1.0.3.jar, /tmp/sbt_7d2cae42/target/b74530de/scala-reflect-2.11.8.jar, /tmp/sbt_7d2cae42/target/9c64274b/xz-1.5.jar, /tmp/sbt_7d2cae42/target/3605eb43/spark-network-common_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/66b46772/commons-compiler-3.0.0.jar, /tmp/sbt_7d2cae42/target/22a95e10/hbase-common-1.0.3.jar, /tmp/sbt_7d2cae42/target/5592374f/oro-2.0.8.jar, /tmp/sbt_7d2cae42/target/1c1efaa3/hadoop-mapreduce-client-app-2.6.5.jar, /tmp/sbt_7d2cae42/target/6f3dc823/jackson-module-paranamer-2.8.7.jar, /tmp/sbt_7d2cae42/target/40719ea6/jsr305-1.3.9.jar, /tmp/sbt_7d2cae42/target/4e3c3796/json4s-ast_2.11-3.2.11.jar, /tmp/sbt_7d2cae42/target/e5642fe0/jetty-util-6.1.26.jar, /tmp/sbt_7d2cae42/target/a698750c/commons-compress-1.8.1.jar, /tmp/sbt_7d2cae42/target/87c0ea80/objenesis-2.1.jar) ) 17/11/27 14:42:55 WARN TaskSetManager: Lost task 0.0 in stage 67.0 (TID 67, localhost, executor driver): UnknownReason 17/11/27 14:42:55 ERROR TaskSetManager: Task 0 in stage 67.0 failed 1 times; aborting job

slim-azaiz commented Nov 27, 2017

@fosskers yes it works but i got these errors : root = sun.misc.Launcher$AppClassLoader@677327b6 cp = Set(/tmp/sbt_7d2cae42/target/a10732c7/protobuf-java-2.5.0.jar, /tmp/sbt_7d2cae42/target/7a9adf97/jersey-media-jaxb-2.22.2.jar, /tmp/sbt_7d2cae42/target/23807d9f/hadoop-yarn-server-common-2.6.5.jar, /tmp/sbt_7d2cae42/target/4ed2b2d4/osgi-resource-locator-1.0.1.jar, /tmp/sbt_7d2cae42/target/b67c76af/scala-xml_2.11-1.0.4.jar, /tmp/sbt_7d2cae42/target/ed364e77/metrics-jvm-3.1.2.jar, /tmp/sbt_7d2cae42/target/35a3c69e/guava-11.0.2.jar, /tmp/sbt_7d2cae42/target/3940343e/hbase-protocol-1.0.3.jar, /tmp/sbt_7d2cae42/target/2973d150/junit-4.12.jar, /tmp/sbt_7d2cae42/target/07ce5e98/cglib-2.2.1-v20090111.jar, /tmp/sbt_7d2cae42/target/65fbf0c7/jets3t-0.9.3.jar, /tmp/sbt_7d2cae42/target/9781746a/netty-all-4.0.43.Final.jar, /tmp/sbt_7d2cae42/target/6925c778/disruptor-3.3.0.jar, /tmp/sbt_7d2cae42/target/5bd604c1/parquet-column-1.8.2.jar, /tmp/sbt_7d2cae42/target/f37255e3/hbase-server-1.0.3.jar, /tmp/sbt_7d2cae42/target/534d72d2/jackson-jaxrs-1.9.13.jar, /tmp/sbt_7d2cae42/target/49f4a68d/hbase-prefix-tree-1.0.3.jar, /tmp/sbt_7d2cae42/target/8dbece93/py4j-0.10.4.jar, /tmp/sbt_7d2cae42/target/a6b11447/findbugs-annotations-1.3.9-1.jar, /tmp/sbt_7d2cae42/target/75a8bd7c/jackson-module-scala_2.11-2.8.7.jar, /tmp/sbt_7d2cae42/target/3cbb2bf5/chill_2.11-0.8.0.jar, /tmp/sbt_7d2cae42/target/104e9c2b/javax.ws.rs-api-2.0.1.jar, /tmp/sbt_7d2cae42/target/964cd741/commons-httpclient-3.1.jar, /tmp/sbt_7d2cae42/target/4d4dde44/spark-catalyst_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/485de3a2/activation-1.1.1.jar, /tmp/sbt_7d2cae42/target/07993c12/commons-net-2.2.jar, /tmp/sbt_7d2cae42/target/9ca7d55d/netty-3.10.5.Final.jar, /tmp/sbt_7d2cae42/target/a510bccf/spark-sketch_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/619eba74/paranamer-2.8.jar, /tmp/sbt_7d2cae42/target/1c16e4e4/apacheds-kerberos-codec-2.0.0-M15.jar, /tmp/sbt_7d2cae42/target/479c1e06/javax.annotation-api-1.2.jar, /tmp/sbt_7d2cae42/target/07407d69/jackson-annotations-2.8.7.jar, /tmp/sbt_7d2cae42/target/9ab8afcc/bcprov-jdk15on-1.51.jar, /tmp/sbt_7d2cae42/target/8906a685/common-config-3.3.1.jar, /tmp/sbt_7d2cae42/target/1df2c042/commons-el-1.0.jar, /tmp/sbt_7d2cae42/target/139535a6/slf4j-api-1.7.21.jar, /tmp/sbt_7d2cae42/target/d8f23aff/avro-mapred-1.7.7-hadoop2.jar, /tmp/sbt_7d2cae42/target/fe1285c9/scala-compiler-2.11.8.jar, /tmp/sbt_7d2cae42/target/ae23df9c/jamon-runtime-2.3.1.jar, /tmp/sbt_7d2cae42/target/6975da39/javax.inject-1.jar, /tmp/sbt_7d2cae42/target/6b7c052b/spark-unsafe_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/8341846f/jersey-core-1.9.jar, /tmp/sbt_7d2cae42/target/a90a8719/kafka-schema-registry-client-3.3.1.jar, /tmp/sbt_7d2cae42/target/7ab1c85a/chill-java-0.8.0.jar, /tmp/sbt_7d2cae42/target/0ef148a1/janino-3.0.0.jar, /tmp/sbt_7d2cae42/target/f91b7a4a/httpcore-4.3.3.jar, /tmp/sbt_7d2cae42/target/ff07b5f1/minlog-1.3.0.jar, /tmp/sbt_7d2cae42/target/88d9e57e/metrics-json-3.1.2.jar, /tmp/sbt_7d2cae42/target/23101cc7/json4s-jackson_2.11-3.2.11.jar, /tmp/sbt_7d2cae42/target/f82c035c/metrics-core-2.2.0.jar, /tmp/sbt_7d2cae42/target/8bd0f95e/hadoop-hdfs-2.6.5.jar, /tmp/sbt_7d2cae42/target/4a6507aa/java-xmlbuilder-1.0.jar, /tmp/sbt_7d2cae42/target/5ede3e5f/jersey-server-2.22.2.jar, /tmp/sbt_7d2cae42/job-1/target/403713c1/aziz_2.11-0.1.jar, /tmp/sbt_7d2cae42/target/ddd5a8bc/scala-library-2.11.8.jar, /tmp/sbt_7d2cae42/target/7f567c6b/hadoop-yarn-api-2.6.5.jar, /tmp/sbt_7d2cae42/target/4bfc12ad/commons-logging-1.2.jar, /tmp/sbt_7d2cae42/target/5e6486ff/api-asn1-api-1.0.0-M20.jar, /tmp/sbt_7d2cae42/target/9add0585/mail-1.4.7.jar, /tmp/sbt_7d2cae42/target/96ca5528/jasper-runtime-5.5.23.jar, /tmp/sbt_7d2cae42/target/a60a5e99/gson-2.2.4.jar, /tmp/sbt_7d2cae42/target/22351b44/hadoop-mapreduce-client-common-2.6.5.jar, /tmp/sbt_7d2cae42/target/7d2e2cea/base64-2.3.8.jar, /tmp/sbt_7d2cae42/target/32cadde2/commons-configuration-1.6.jar, /tmp/sbt_7d2cae42/target/dc366f22/hadoop-auth-2.6.5.jar, /tmp/sbt_7d2cae42/target/7422c693/jersey-guava-2.22.2.jar, /tmp/sbt_7d2cae42/target/87d4c44d/json4s-core_2.11-3.2.11.jar, /tmp/sbt_7d2cae42/target/48d92871/snappy-java-1.1.2.6.jar, /tmp/sbt_7d2cae42/target/3e1495b0/compress-lzf-1.0.3.jar, /tmp/sbt_7d2cae42/target/47bf147f/mx4j-3.0.2.jar, /tmp/sbt_7d2cae42/target/9d84f15f/guice-3.0.jar, /tmp/sbt_7d2cae42/target/e6b04760/RoaringBitmap-0.5.11.jar, /tmp/sbt_7d2cae42/target/dc6a73fd/commons-digester-1.8.jar, /tmp/sbt_7d2cae42/target/71c61c84/apacheds-i18n-2.0.0-M15.jar, /tmp/sbt_7d2cae42/target/8736b0fc/curator-recipes-2.6.0.jar, /tmp/sbt_7d2cae42/target/d90487b9/jersey-container-servlet-2.22.2.jar, /tmp/sbt_7d2cae42/target/5abe4c24/ivy-2.4.0.jar, /tmp/sbt_7d2cae42/target/034e4807/jcl-over-slf4j-1.7.16.jar, /tmp/sbt_7d2cae42/target/91e3146d/avro-1.8.2.jar, /tmp/sbt_7d2cae42/target/707350a2/leveldbjni-all-1.8.jar, /tmp/sbt_7d2cae42/target/d6337b0d/stax-api-1.0-2.jar, /tmp/sbt_7d2cae42/target/4b95f489/commons-codec-1.10.jar, /tmp/sbt_7d2cae42/target/0235ba8b/aopalliance-1.0.jar, /tmp/sbt_7d2cae42/target/7938f66b/commons-crypto-1.0.0.jar, /tmp/sbt_7d2cae42/target/1a01a2a1/jettison-1.1.jar, /tmp/sbt_7d2cae42/target/6c6c702c/commons-lang3-3.5.jar, /tmp/sbt_7d2cae42/target/7238b064/slf4j-log4j12-1.7.21.jar, /tmp/sbt_7d2cae42/target/8ad72fe3/commons-collections-3.2.2.jar, /tmp/sbt_7d2cae42/target/f1a3bc53/parquet-jackson-1.8.2.jar, /tmp/sbt_7d2cae42/target/5d4c38ae/kafka_2.11-0.8.2.1.jar, /tmp/sbt_7d2cae42/target/3d5e856d/aopalliance-repackaged-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/8e73d3de/hbase-annotations-1.0.3.jar, /tmp/sbt_7d2cae42/target/c157def1/asm-3.1.jar, /tmp/sbt_7d2cae42/target/77b2fb4c/spark-avro_2.11-4.0.0.jar, /tmp/sbt_7d2cae42/target/4c47155e/httpclient-4.3.6.jar, /tmp/sbt_7d2cae42/target/8613ae82/validation-api-1.1.0.Final.jar, /tmp/sbt_7d2cae42/target/2008b781/hadoop-mapreduce-client-shuffle-2.6.5.jar, /tmp/sbt_7d2cae42/target/9ef74113/kafka-clients-0.8.2.1.jar, /tmp/sbt_7d2cae42/target/4b81d62b/spark-network-shuffle_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/a6a3d493/javax.inject-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/29262f09/hadoop-annotations-2.6.5.jar, /tmp/sbt_7d2cae42/target/1f08024e/joni-2.1.2.jar, /tmp/sbt_7d2cae42/target/1209b898/jersey-common-2.22.2.jar, /tmp/sbt_7d2cae42/target/0e1222db/scalap-2.11.8.jar, /tmp/sbt_7d2cae42/target/b71a4668/pyrolite-4.13.jar, /tmp/sbt_7d2cae42/target/370b5cff/common-utils-3.3.1.jar, /tmp/sbt_7d2cae42/target/c56c5d4e/spark-sql_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/f04ea614/hadoop-yarn-client-2.6.5.jar, /tmp/sbt_7d2cae42/target/c3210db2/spark-streaming-kafka-0-8_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/fd071844/univocity-parsers-2.2.1.jar, /tmp/sbt_7d2cae42/target/f73606e7/htrace-core-3.1.0-incubating.jar, /tmp/sbt_7d2cae42/target/211fe2e6/parquet-hadoop-1.8.2.jar, /tmp/sbt_7d2cae42/target/7369d653/scala-parser-combinators_2.11-1.0.4.jar, /tmp/sbt_7d2cae42/target/205fe37a/unused-1.0.0.jar, /tmp/sbt_7d2cae42/target/b1b6ea3b/commons-io-2.4.jar, /tmp/sbt_7d2cae42/target/0988e998/hadoop-mapreduce-client-jobclient-2.6.5.jar, /tmp/sbt_7d2cae42/target/aeb3021c/jaxb-api-2.2.2.jar, /tmp/sbt_7d2cae42/target/756b3fac/jsp-api-2.1-6.1.14.jar, /tmp/sbt_7d2cae42/target/4877b85d/commons-math-2.2.jar, /tmp/sbt_7d2cae42/target/38383ecc/hadoop-mapreduce-client-core-2.6.5.jar, /tmp/sbt_7d2cae42/target/977d6b42/kryo-shaded-3.0.3.jar, /tmp/sbt_7d2cae42/target/2d5b546c/jul-to-slf4j-1.7.16.jar, /tmp/sbt_7d2cae42/target/3ac44a86/commons-math3-3.4.1.jar, /tmp/sbt_7d2cae42/target/1712fff0/jersey-client-2.22.2.jar, /tmp/sbt_7d2cae42/target/1017432e/hk2-api-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/50d7a8e2/hbase-client-1.0.3.jar, /tmp/sbt_7d2cae42/target/1c36c81e/jackson-databind-2.8.4.jar, /tmp/sbt_7d2cae42/target/d9a09f77/javassist-3.18.1-GA.jar, /tmp/sbt_7d2cae42/target/c54d4b5a/zkclient-0.10.jar, /tmp/sbt_7d2cae42/target/9998f8cf/stream-2.7.0.jar, /tmp/sbt_7d2cae42/target/a871abf0/api-util-1.0.0-M20.jar, /tmp/sbt_7d2cae42/target/3c304d70/jackson-core-asl-1.9.13.jar, /tmp/sbt_7d2cae42/target/523114bb/kafka-avro-serializer-3.3.1.jar, /tmp/sbt_7d2cae42/target/542df0ff/spark-tags_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/3a6ea7cc/jersey-server-1.9.jar, /tmp/sbt_7d2cae42/target/90b215f4/xml-apis-1.3.04.jar, /tmp/sbt_7d2cae42/target/60367999/jetty-sslengine-6.1.26.jar, /tmp/sbt_7d2cae42/target/c708bb25/lz4-1.3.0.jar, /tmp/sbt_7d2cae42/target/7bc7e49d/xercesImpl-2.9.1.jar, /tmp/sbt_7d2cae42/target/d82554ef/xmlenc-0.52.jar, /tmp/sbt_7d2cae42/target/08eebdbb/zookeeper-3.4.10.jar, /tmp/sbt_7d2cae42/target/aacce184/hk2-utils-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/33fa45fd/jcodings-1.0.8.jar, /tmp/sbt_7d2cae42/target/b9007c35/curator-client-2.6.0.jar, /tmp/sbt_7d2cae42/target/81a699c3/curator-framework-2.6.0.jar, /tmp/sbt_7d2cae42/target/c318b7a4/hadoop-yarn-common-2.6.5.jar, /tmp/sbt_7d2cae42/target/f1e6dbc3/spark-core_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/3cd63d07/javax.servlet-api-3.1.0.jar, /tmp/sbt_7d2cae42/target/1451fc3e/hk2-locator-2.4.0-b34.jar, /tmp/sbt_7d2cae42/target/f317ae40/spark-launcher_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/336cc5c7/spark-streaming_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/99a18e9a/jline-0.9.94.jar, /tmp/sbt_7d2cae42/target/1ee2f2be/jackson-mapper-asl-1.9.13.jar, /tmp/sbt_7d2cae42/target/775c6ba9/hadoop-common-2.6.5.jar, /tmp/sbt_7d2cae42/target/42a25dc3/hamcrest-core-1.3.jar, /tmp/sbt_7d2cae42/target/5675fd96/commons-beanutils-1.7.0.jar, /tmp/sbt_7d2cae42/target/212c534c/jersey-container-servlet-core-2.22.2.jar, /tmp/sbt_7d2cae42/target/8b46f39c/jackson-core-2.8.7.jar, /tmp/sbt_7d2cae42/target/2bf96b7a/commons-cli-1.2.jar, /tmp/sbt_7d2cae42/target/d7c75374/hadoop-client-2.6.5.jar, /tmp/sbt_7d2cae42/target/2609e36f/antlr4-runtime-4.5.3.jar, /tmp/sbt_7d2cae42/target/0ce1edb9/commons-lang-2.6.jar, /tmp/sbt_7d2cae42/target/15a68399/metrics-graphite-3.1.2.jar, /tmp/sbt_7d2cae42/target/2a3e3137/parquet-format-2.3.1.jar, /tmp/sbt_7d2cae42/target/175dc721/commons-beanutils-core-1.8.0.jar, /tmp/sbt_7d2cae42/target/b754e613/avro-ipc-1.7.7.jar, /tmp/sbt_7d2cae42/target/765ab4bb/parquet-encoding-1.8.2.jar, /tmp/sbt_7d2cae42/target/3a1df1e8/jsp-2.1-6.1.14.jar, /tmp/sbt_7d2cae42/target/1ef0eeef/avro-ipc-1.7.7-tests.jar, /tmp/sbt_7d2cae42/target/6d35bd7f/jasper-compiler-5.5.23.jar, /tmp/sbt_7d2cae42/target/224f03af/metrics-core-3.1.2.jar, /tmp/sbt_7d2cae42/target/a413bb5a/xbean-asm5-shaded-4.4.jar, /tmp/sbt_7d2cae42/target/e3480072/jackson-xc-1.9.13.jar, /tmp/sbt_7d2cae42/target/5af35056/log4j-1.2.17.jar, /tmp/sbt_7d2cae42/target/fd7cc70f/parquet-common-1.8.2.jar, /tmp/sbt_7d2cae42/target/d7461828/htrace-core-3.0.4.jar, /tmp/sbt_7d2cae42/target/a2fd5fbc/spark-hbase-connector_2.10-1.0.3.jar, /tmp/sbt_7d2cae42/target/b74530de/scala-reflect-2.11.8.jar, /tmp/sbt_7d2cae42/target/9c64274b/xz-1.5.jar, /tmp/sbt_7d2cae42/target/3605eb43/spark-network-common_2.11-2.2.0.jar, /tmp/sbt_7d2cae42/target/66b46772/commons-compiler-3.0.0.jar, /tmp/sbt_7d2cae42/target/22a95e10/hbase-common-1.0.3.jar, /tmp/sbt_7d2cae42/target/5592374f/oro-2.0.8.jar, /tmp/sbt_7d2cae42/target/1c1efaa3/hadoop-mapreduce-client-app-2.6.5.jar, /tmp/sbt_7d2cae42/target/6f3dc823/jackson-module-paranamer-2.8.7.jar, /tmp/sbt_7d2cae42/target/40719ea6/jsr305-1.3.9.jar, /tmp/sbt_7d2cae42/target/4e3c3796/json4s-ast_2.11-3.2.11.jar, /tmp/sbt_7d2cae42/target/e5642fe0/jetty-util-6.1.26.jar, /tmp/sbt_7d2cae42/target/a698750c/commons-compress-1.8.1.jar, /tmp/sbt_7d2cae42/target/87c0ea80/objenesis-2.1.jar) ) 17/11/27 14:42:55 WARN TaskSetManager: Lost task 0.0 in stage 67.0 (TID 67, localhost, executor driver): UnknownReason 17/11/27 14:42:55 ERROR TaskSetManager: Task 0 in stage 67.0 failed 1 times; aborting job

@fosskers

This comment has been minimized.

Show comment
Hide comment
@fosskers

fosskers Nov 27, 2017

Was there an actual Exception thrown?

fosskers commented Nov 27, 2017

Was there an actual Exception thrown?

@ravwojdyla

This comment has been minimized.

Show comment
Hide comment
@ravwojdyla

ravwojdyla commented Dec 21, 2017

PTAL: sbt/zinc#473

@eed3si9n eed3si9n added this to the 1.1.0 milestone Dec 22, 2017

@eed3si9n

This comment has been minimized.

Show comment
Hide comment
@eed3si9n

eed3si9n Dec 22, 2017

Member

Zinc fix is now merged. Please try sbt 1.1.0-RC3 sbt 1.1.0-RC4.

Member

eed3si9n commented Dec 22, 2017

Zinc fix is now merged. Please try sbt 1.1.0-RC3 sbt 1.1.0-RC4.

@eed3si9n eed3si9n closed this Dec 22, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment