[ADAM-2145] Add Spark 2.4.3 and Scala 2.12 to Jenkins build #2149

heuermh · 2019-05-13T21:31:57Z

coveralls · 2019-05-13T21:53:24Z

Coverage remained the same at 78.879% when pulling 6bed13d on heuermh:travis-spark-2.4-scala-2.12 into 9f17a0f on bigdatagenomics:master.

AmplabJenkins · 2019-05-13T22:09:23Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2984/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 98085e3 # timeout=10Checking out Revision 98085e3 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 98085e3e948dc5cd7a5ee5ca74689e44e96b2038First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-14T00:36:31Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2985/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 0d6565b # timeout=10Checking out Revision 0d6565b (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 0d6565b036217e99af3454b3fcb49a90c1ff6ee0First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-14T19:10:32Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2986/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 3718c8d # timeout=10Checking out Revision 3718c8d (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 3718c8d21eb71b332d4cb15a65a62054d27f210fFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-14T20:08:00Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2987/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 26bf669 # timeout=10Checking out Revision 26bf669 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 26bf66931a4c4bfb5f49f27d5dc0967ca3493575First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-15T16:00:25Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2988/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 08944b4 # timeout=10Checking out Revision 08944b4 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 08944b44bddbba2bea95b14b8e9631eb95c0205aFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-15T17:23:13Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2989/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 6ee9ab9 # timeout=10Checking out Revision 6ee9ab9 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 6ee9ab951e276e23f45f21deb458d206f12d3d36First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-15T19:24:22Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2990/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 1f4f66d # timeout=10Checking out Revision 1f4f66d (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 1f4f66dcee6a8a4c72bbf0e93421e60d60b54b5eFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-15T20:43:40Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2991/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 8f09a55 # timeout=10Checking out Revision 8f09a55 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 8f09a5504fee4f06243a20b42b534b17aef33d46First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-15T22:12:19Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2992/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 646d138 # timeout=10Checking out Revision 646d138 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 646d138fa281f24592dba23dfac1851c61b354f8First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

akmorrow13 · 2019-05-16T01:08:13Z

pom.xml

@@ -588,6 +588,11 @@
        <artifactId>adam-codegen-spark2_${scala.version.prefix}</artifactId>
        <version>${project.version}</version>
      </dependency>
+      <dependency>
+        <groupId>com.thoughtworks.paranamer</groupId>


What are you using this for?

More classpath woes
https://issues.apache.org/jira/browse/SPARK-26146
https://issues.apache.org/jira/browse/SPARK-26583
https://issues.apache.org/jira/browse/SPARK-22128

It's been very frustrating dealing with the Spark folks, they keep going fast and breaking things without cleaning up any of the dependency mess left in the wake.

"But it all seems to work in a quick test."

Good thing there aren't any projects downstream of Spark.

AmplabJenkins · 2019-05-19T01:05:43Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2993/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 18b4c43 # timeout=10Checking out Revision 18b4c43 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 18b4c43132ac49c2c3c64bf230086a74187c794dFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-19T01:47:48Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2994/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 9893847 # timeout=10Checking out Revision 9893847 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 9893847a23c61ae931e2881cbda0506ec81b2eb1First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-19T15:00:03Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2995/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 10cacb9 # timeout=10Checking out Revision 10cacb9 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 10cacb972bef001742bbacf130ca5e98052b42ebFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-19T16:58:14Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2996/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 39375fa # timeout=10Checking out Revision 39375fa (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 39375fa95d1855d87d065faee72e616c2a0dac27First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-19T23:50:49Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2997/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains c49ba42 # timeout=10Checking out Revision c49ba42 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f c49ba426e1993635b92693e7cf4ecf67ec8cd43bFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-20T16:49:05Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2998/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 5906981 # timeout=10Checking out Revision 5906981 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 5906981ec3080b590e73d868753042b644cf1985First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2019-05-20T19:42:28Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2999/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 124c553 # timeout=10Checking out Revision 124c553 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 124c553770be973e6f9fc5dca2bbeb8cd73b3ea9First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

heuermh · 2019-05-21T18:23:19Z

Jenkins, retest this please

AmplabJenkins · 2019-05-21T19:08:07Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3000/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > git rev-parse origin/pr/2149/merge^{commit} # timeout=10 > git branch -a -v --no-abbrev --contains 124c553 # timeout=10Checking out Revision 124c553 (origin/pr/2149/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 124c553770be973e6f9fc5dca2bbeb8cd73b3ea9First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.5,2.12,2.4.3,ubuntuTriggering ADAM-prb ? 2.7.5,2.11,2.4.3,ubuntuADAM-prb ? 2.7.5,2.12,2.4.3,ubuntu completed with result FAILUREADAM-prb ? 2.7.5,2.11,2.4.3,ubuntu completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

heuermh · 2019-05-21T19:41:37Z

@fnothaft I'm very confused by the most recent Jenkins test failure.

It says that avro GenericRecord is missing from the classpath

Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/avro/generic/GenericRecord
	at org.apache.spark.serializer.KryoSerializer.newKryo(KryoSerializer.scala:123)
	at org.apache.spark.serializer.KryoSerializerInstance.borrowKryo(KryoSerializer.scala:324)
	at org.apache.spark.serializer.KryoSerializerInstance.<init>(KryoSerializer.scala:309)
	at org.apache.spark.serializer.KryoSerializer.newInstance(KryoSerializer.scala:218)
	at org.apache.spark.broadcast.TorrentBroadcast$.blockifyObject(TorrentBroadcast.scala:288)
	at org.apache.spark.broadcast.TorrentBroadcast.writeBlocks(TorrentBroadcast.scala:127)
	at org.apache.spark.broadcast.TorrentBroadcast.<init>(TorrentBroadcast.scala:88)
	at org.apache.spark.broadcast.TorrentBroadcastFactory.newBroadcast(TorrentBroadcastFactory.scala:34)
	at org.apache.spark.broadcast.BroadcastManager.newBroadcast(BroadcastManager.scala:62)
	at org.apache.spark.SparkContext.broadcast(SparkContext.scala:1489)
	at org.apache.spark.rdd.NewHadoopRDD.<init>(NewHadoopRDD.scala:79)
	at org.apache.spark.SparkContext.$anonfun$newAPIHadoopFile$2(SparkContext.scala:1160)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151)
	at org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:112)
	at org.apache.spark.SparkContext.withScope(SparkContext.scala:699)
	at org.apache.spark.SparkContext.newAPIHadoopFile(SparkContext.scala:1146)
	at org.bdgenomics.adam.rdd.ADAMContext.$anonfun$loadBam$1(ADAMContext.scala:1583)
...
Caused by: java.lang.ClassNotFoundException: org.apache.avro.generic.GenericRecord
	at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
	... 43 more

but that class is present in the ADAM assembly jar

$ jar tvf adam-assembly/target/adam-assembly-spark2_2.12-0.27.0-SNAPSHOT.jar \
  | grep GenericRecord

   287 Tue May 21 14:15:12 CDT 2019 org/apache/avro/generic/GenericRecord.class
  5860 Tue May 21 14:15:12 CDT 2019 org/apache/avro/generic/GenericRecordBuilder.class

as avro-1.8.2.jar is pulled in in compile scope via bdg-formats 0.12.0

$ mvn dependency:tree
...
[INFO] --- maven-dependency-plugin:2.8:tree (default-cli) @ adam-assembly-spark2_2.12 ---
[INFO] org.bdgenomics.adam:adam-assembly-spark2_2.12:jar:0.27.0-SNAPSHOT
[INFO] \- org.bdgenomics.adam:adam-cli-spark2_2.12:jar:0.27.0-SNAPSHOT:compile
...
[INFO]    +- org.bdgenomics.bdg-formats:bdg-formats:jar:0.12.0:compile
[INFO]    |  \- org.apache.avro:avro:jar:1.8.2:compile

I can also reproduce locally.

AmplabJenkins · 2019-05-21T22:25:41Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/3001/
Test PASSed.

Add Spark 2.4.3 and Scala 2.12 to Jenkins build.

ce12af9

heuermh added this to the 0.27.0 milestone May 13, 2019

heuermh added 2 commits May 13, 2019 18:48

Fix for Jenkins failures.

cb02e88

remove move_to_spark_2.sh check

fb90fd7

move back to scala 2.11

0babacd

move before git test

45cb2d8

still not in correct place

5285aaa

Spark 2.4.3 needs special case for Scala 2.12

c074a30

export SPARK_HOME correctly

35fa50a

download hadoop and add to classpath

366efa3

try adding explicit dependency on paranamer 2.8

75e0d89

heuermh requested review from fnothaft and akmorrow13 May 16, 2019 00:27

akmorrow13 reviewed May 16, 2019

View reviewed changes

akmorrow13 approved these changes May 16, 2019

View reviewed changes

try specifying spark.executor.memory

1e17c19

moar

39fbad5

try PYSPARK_SUBMIT_ARGS in Makefile

78442f3

try again

6c825b5

fix conf syntax

528c3f5

move conf to jenkins-test

dd49f66

remove references to avro 1.7.x

8ba9a92

install avro 1.8.x

6bed13d

heuermh merged commit 6045e41 into bigdatagenomics:master May 22, 2019

heuermh deleted the travis-spark-2.4-scala-2.12 branch May 22, 2019 00:15

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ADAM-2145] Add Spark 2.4.3 and Scala 2.12 to Jenkins build #2149

[ADAM-2145] Add Spark 2.4.3 and Scala 2.12 to Jenkins build #2149

heuermh commented May 13, 2019

coveralls commented May 13, 2019 •

edited

Loading

AmplabJenkins commented May 13, 2019

AmplabJenkins commented May 14, 2019

AmplabJenkins commented May 14, 2019

AmplabJenkins commented May 14, 2019

AmplabJenkins commented May 15, 2019

AmplabJenkins commented May 15, 2019

AmplabJenkins commented May 15, 2019

AmplabJenkins commented May 15, 2019

AmplabJenkins commented May 15, 2019

akmorrow13 May 16, 2019

heuermh May 16, 2019 •

edited

Loading

akmorrow13 May 16, 2019

heuermh May 16, 2019 •

edited

Loading

AmplabJenkins commented May 19, 2019

AmplabJenkins commented May 19, 2019

AmplabJenkins commented May 19, 2019

AmplabJenkins commented May 19, 2019

AmplabJenkins commented May 19, 2019

AmplabJenkins commented May 20, 2019

AmplabJenkins commented May 20, 2019

heuermh commented May 21, 2019

AmplabJenkins commented May 21, 2019

heuermh commented May 21, 2019 •

edited

Loading

AmplabJenkins commented May 21, 2019

[ADAM-2145] Add Spark 2.4.3 and Scala 2.12 to Jenkins build #2149

[ADAM-2145] Add Spark 2.4.3 and Scala 2.12 to Jenkins build #2149

Conversation

heuermh commented May 13, 2019

coveralls commented May 13, 2019 • edited Loading

AmplabJenkins commented May 13, 2019

Build result: FAILURE

AmplabJenkins commented May 14, 2019

Build result: FAILURE

AmplabJenkins commented May 14, 2019

Build result: FAILURE

AmplabJenkins commented May 14, 2019

Build result: FAILURE

AmplabJenkins commented May 15, 2019

Build result: FAILURE

AmplabJenkins commented May 15, 2019

Build result: FAILURE

AmplabJenkins commented May 15, 2019

Build result: FAILURE

AmplabJenkins commented May 15, 2019

Build result: FAILURE

AmplabJenkins commented May 15, 2019

Build result: FAILURE

akmorrow13 May 16, 2019

Choose a reason for hiding this comment

heuermh May 16, 2019 • edited Loading

Choose a reason for hiding this comment

akmorrow13 May 16, 2019

Choose a reason for hiding this comment

heuermh May 16, 2019 • edited Loading

Choose a reason for hiding this comment

AmplabJenkins commented May 19, 2019

Build result: FAILURE

AmplabJenkins commented May 19, 2019

Build result: FAILURE

AmplabJenkins commented May 19, 2019

Build result: FAILURE

AmplabJenkins commented May 19, 2019

Build result: FAILURE

AmplabJenkins commented May 19, 2019

Build result: FAILURE

AmplabJenkins commented May 20, 2019

Build result: FAILURE

AmplabJenkins commented May 20, 2019

Build result: FAILURE

heuermh commented May 21, 2019

AmplabJenkins commented May 21, 2019

Build result: FAILURE

heuermh commented May 21, 2019 • edited Loading

AmplabJenkins commented May 21, 2019

coveralls commented May 13, 2019 •

edited

Loading

heuermh May 16, 2019 •

edited

Loading

heuermh May 16, 2019 •

edited

Loading

heuermh commented May 21, 2019 •

edited

Loading