New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update dependency and plugin versions #1360

Merged
merged 1 commit into from Apr 6, 2017

Conversation

Projects
6 participants
@heuermh
Member

heuermh commented Jan 20, 2017

Notes to self:

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jan 20, 2017

Member

Nice! We should probably hold on going to Avro 1.8.2 until they have an official release and not just a RC, no? Otherwise, SGTM. Let me look into that servo dependency...

Member

fnothaft commented Jan 20, 2017

Nice! We should probably hold on going to Avro 1.8.2 until they have an official release and not just a RC, no? Otherwise, SGTM. Let me look into that servo dependency...

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jan 20, 2017

Member

Yeah, I'm still investigating this and will have further commits. I just wanted to have Jenkins confirm things piecemeal for me, cuz running the Jenkins tests on my laptop takes forever.

Member

heuermh commented Jan 20, 2017

Yeah, I'm still investigating this and will have further commits. I just wanted to have Jenkins confirm things piecemeal for me, cuz running the Jenkins tests on my laptop takes forever.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 20, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1745/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 2b5738672a75951f9403dc9e8bdbef143fc96e71 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 2b5738672a75951f9403dc9e8bdbef143fc96e71 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 2b5738672a75951f9403dc9e8bdbef143fc96e71First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Jan 20, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1745/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 2b5738672a75951f9403dc9e8bdbef143fc96e71 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 2b5738672a75951f9403dc9e8bdbef143fc96e71 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 2b5738672a75951f9403dc9e8bdbef143fc96e71First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jan 20, 2017

Member

bumping servo from 0.5.5 to 0.10.0 or 0.12.12 upstream in utils results in compile errors

[INFO] Compiling 22 source files to /Users/heuermh/working/bdg-utils/utils-metrics/target/2.10.6/classes at 1484931081411
[WARNING] warning: Class org.slf4j.Logger not found - continuing with a stub.
[ERROR] /Users/heuermh/working/bdg-utils/utils-metrics/src/main/scala/org/bdgenomics/utils/instrumentation/Metrics.scala:323: error: class StringMonitor needs to be abstract, since method getValue in trait Monitor of type (x$1: Int)String is not defined
[ERROR]   private class StringMonitor(name: String, value: String, tags: Tag*) extends Monitor[String] {
[ERROR]                 ^
[ERROR] /Users/heuermh/working/bdg-utils/utils-metrics/src/main/scala/org/bdgenomics/utils/instrumentation/ServoTimer.scala:44: error: class ServoTimer needs to be abstract, since method getValue in trait Monitor of type (x$1: Int)Object is not defined
[ERROR] class ServoTimer(name: String, @transient tags: Tag*) extends ConfigurableMonitor(name, tags)
[ERROR]       ^
[ERROR] /Users/heuermh/working/bdg-utils/utils-metrics/src/main/scala/org/bdgenomics/utils/instrumentation/ServoTimer.scala:161: error: class LongValueStatMonitor needs to be abstract, since method getValue in trait Monitor of type (x$1: Int)Long is not defined
[ERROR]   private class LongValueStatMonitor(name: String, @transient tags: Seq[Tag], function: () => Long)
[ERROR]                 ^
[ERROR] /Users/heuermh/working/bdg-utils/utils-metrics/src/main/scala/org/bdgenomics/utils/instrumentation/ServoTimer.scala:168: error: class ConditionalGauge needs to be abstract, since method getValue in trait Monitor of type (x$1: Int)Long is not defined
[ERROR]   private class ConditionalGauge(name: String, @transient tags: Seq[Tag], condition: (Long, Long) => Boolean)
[ERROR]                 ^
[WARNING] one warning found
[ERROR] four errors found
Member

heuermh commented Jan 20, 2017

bumping servo from 0.5.5 to 0.10.0 or 0.12.12 upstream in utils results in compile errors

[INFO] Compiling 22 source files to /Users/heuermh/working/bdg-utils/utils-metrics/target/2.10.6/classes at 1484931081411
[WARNING] warning: Class org.slf4j.Logger not found - continuing with a stub.
[ERROR] /Users/heuermh/working/bdg-utils/utils-metrics/src/main/scala/org/bdgenomics/utils/instrumentation/Metrics.scala:323: error: class StringMonitor needs to be abstract, since method getValue in trait Monitor of type (x$1: Int)String is not defined
[ERROR]   private class StringMonitor(name: String, value: String, tags: Tag*) extends Monitor[String] {
[ERROR]                 ^
[ERROR] /Users/heuermh/working/bdg-utils/utils-metrics/src/main/scala/org/bdgenomics/utils/instrumentation/ServoTimer.scala:44: error: class ServoTimer needs to be abstract, since method getValue in trait Monitor of type (x$1: Int)Object is not defined
[ERROR] class ServoTimer(name: String, @transient tags: Tag*) extends ConfigurableMonitor(name, tags)
[ERROR]       ^
[ERROR] /Users/heuermh/working/bdg-utils/utils-metrics/src/main/scala/org/bdgenomics/utils/instrumentation/ServoTimer.scala:161: error: class LongValueStatMonitor needs to be abstract, since method getValue in trait Monitor of type (x$1: Int)Long is not defined
[ERROR]   private class LongValueStatMonitor(name: String, @transient tags: Seq[Tag], function: () => Long)
[ERROR]                 ^
[ERROR] /Users/heuermh/working/bdg-utils/utils-metrics/src/main/scala/org/bdgenomics/utils/instrumentation/ServoTimer.scala:168: error: class ConditionalGauge needs to be abstract, since method getValue in trait Monitor of type (x$1: Int)Long is not defined
[ERROR]   private class ConditionalGauge(name: String, @transient tags: Seq[Tag], condition: (Long, Long) => Boolean)
[ERROR]                 ^
[WARNING] one warning found
[ERROR] four errors found
@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh
Member

heuermh commented Jan 20, 2017

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 20, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1746/
Test PASSed.

AmplabJenkins commented Jan 20, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1746/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 20, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1747/
Test PASSed.

AmplabJenkins commented Jan 20, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1747/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 23, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1749/
Test PASSed.

AmplabJenkins commented Jan 23, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1749/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 23, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1751/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 1722b2f694cda28eca8fbee12830993b79e738ca # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 1722b2f694cda28eca8fbee12830993b79e738ca (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 1722b2f694cda28eca8fbee12830993b79e738caFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Jan 23, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1751/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 1722b2f694cda28eca8fbee12830993b79e738ca # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 1722b2f694cda28eca8fbee12830993b79e738ca (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 1722b2f694cda28eca8fbee12830993b79e738caFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jan 23, 2017

Member

I also see the exception in the Jenkins build failure logs when running adam-submit locally.

java.lang.NoSuchMethodError: org.apache.avro.Schema.getLogicalType()Lorg/apache/avro/LogicalType;
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:178)
	at org.apache.parquet.avro.AvroSchemaConverter.convertUnion(AvroSchemaConverter.java:214)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:171)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:130)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:227)
	at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:124)
	at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:115)
	at org.apache.parquet.avro.AvroWriteSupport.init(AvroWriteSupport.java:117)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:288)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:262)
	at org.apache.spark.rdd.InstrumentedOutputFormat.getRecordWriter(InstrumentedOutputFormat.scala:35)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1030)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1014)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
	at org.apache.spark.scheduler.Task.run(Task.scala:88)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

I'm not sure where this runtime conflict is coming from. The method Schema.getLogicalType() was added to avro post version 1.7.7, and all our dependencies should be at least avro version 1.8.0.

$ mvn dependency:tree | grep avro
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:provided
[INFO] |  |  +- org.apache.avro:avro-ipc:jar:1.7.7:provided
[INFO] |  |  \- org.apache.avro:avro-ipc:jar:tests:1.7.7:provided
[INFO] +- org.apache.avro:avro:jar:1.8.1:compile
[INFO] +- org.apache.parquet:parquet-avro:jar:1.8.2:compile

Oops.

Member

heuermh commented Jan 23, 2017

I also see the exception in the Jenkins build failure logs when running adam-submit locally.

java.lang.NoSuchMethodError: org.apache.avro.Schema.getLogicalType()Lorg/apache/avro/LogicalType;
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:178)
	at org.apache.parquet.avro.AvroSchemaConverter.convertUnion(AvroSchemaConverter.java:214)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:171)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:130)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:227)
	at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:124)
	at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:115)
	at org.apache.parquet.avro.AvroWriteSupport.init(AvroWriteSupport.java:117)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:288)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:262)
	at org.apache.spark.rdd.InstrumentedOutputFormat.getRecordWriter(InstrumentedOutputFormat.scala:35)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1030)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1014)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
	at org.apache.spark.scheduler.Task.run(Task.scala:88)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

I'm not sure where this runtime conflict is coming from. The method Schema.getLogicalType() was added to avro post version 1.7.7, and all our dependencies should be at least avro version 1.8.0.

$ mvn dependency:tree | grep avro
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.7.7:provided
[INFO] |  |  +- org.apache.avro:avro-ipc:jar:1.7.7:provided
[INFO] |  |  \- org.apache.avro:avro-ipc:jar:tests:1.7.7:provided
[INFO] +- org.apache.avro:avro:jar:1.8.1:compile
[INFO] +- org.apache.parquet:parquet-avro:jar:1.8.2:compile

Oops.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 23, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1753/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 0771250a0a40914ca60b6829744bb28ab1a3dafe # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 0771250a0a40914ca60b6829744bb28ab1a3dafe (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 0771250a0a40914ca60b6829744bb28ab1a3dafeFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Jan 23, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1753/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 0771250a0a40914ca60b6829744bb28ab1a3dafe # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 0771250a0a40914ca60b6829744bb28ab1a3dafe (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 0771250a0a40914ca60b6829744bb28ab1a3dafeFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jan 23, 2017

Member

Jenkins, retest this please

Member

heuermh commented Jan 23, 2017

Jenkins, retest this please

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 23, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1754/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 3c907d9 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 3c907d9 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3c907d948e99a2802a4692e82dc3936e947d3f57First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Jan 23, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1754/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 3c907d9 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 3c907d9 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3c907d948e99a2802a4692e82dc3936e947d3f57First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jan 24, 2017

Member

Jenkins, retest this please

Member

heuermh commented Jan 24, 2017

Jenkins, retest this please

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 24, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1757/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 3c907d9 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 3c907d9 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3c907d948e99a2802a4692e82dc3936e947d3f57First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Jan 24, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1757/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 3c907d9 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 3c907d9 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3c907d948e99a2802a4692e82dc3936e947d3f57First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jan 24, 2017

Member

Still not sure what I'm missing here, all the dependency versions are 1.8.1 yet something is running into a 1.7.7 or earlier version of the avro Schema class at runtime

$ mvn dependency:tree | grep avro
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.8.1:provided
[INFO] |  |  \- org.apache.avro:avro-ipc:jar:1.8.1:provided
[INFO] +- org.apache.avro:avro:jar:1.8.1:compile
[INFO] +- org.apache.parquet:parquet-avro:jar:1.8.2:compile
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.8.1:provided
[INFO] |  |  +- org.apache.avro:avro-ipc:jar:1.8.1:provided
[INFO] |  \- org.apache.avro:avro:jar:1.8.1:compile
[INFO] |  +- org.apache.parquet:parquet-avro:jar:1.8.2:compile
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.8.1:provided
[INFO] |  |  +- org.apache.avro:avro-ipc:jar:1.8.1:provided
[INFO] |  \- org.apache.parquet:parquet-avro:jar:1.8.2:compile
[INFO] |  \- org.apache.avro:avro:jar:1.8.1:compile
[INFO] |  |  \- org.apache.parquet:parquet-avro:jar:1.8.2:compile
[INFO] |  |  \- org.apache.avro:avro:jar:1.8.1:compile

(duplicates here because I am running from the parent directory)

Avro 1.7.7 is present in the Spark lib folder

$ ls /usr/local/Cellar/apache-spark/2.1.0/libexec/jars/avro-*
/usr/local/Cellar/apache-spark/2.1.0/libexec/jars/avro-1.7.7.jar
/usr/local/Cellar/apache-spark/2.1.0/libexec/jars/avro-ipc-1.7.7.jar
/usr/local/Cellar/apache-spark/2.1.0/libexec/jars/avro-mapred-1.7.7-hadoop2.jar

Are we not able to fully override these at runtime?

Member

heuermh commented Jan 24, 2017

Still not sure what I'm missing here, all the dependency versions are 1.8.1 yet something is running into a 1.7.7 or earlier version of the avro Schema class at runtime

$ mvn dependency:tree | grep avro
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.8.1:provided
[INFO] |  |  \- org.apache.avro:avro-ipc:jar:1.8.1:provided
[INFO] +- org.apache.avro:avro:jar:1.8.1:compile
[INFO] +- org.apache.parquet:parquet-avro:jar:1.8.2:compile
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.8.1:provided
[INFO] |  |  +- org.apache.avro:avro-ipc:jar:1.8.1:provided
[INFO] |  \- org.apache.avro:avro:jar:1.8.1:compile
[INFO] |  +- org.apache.parquet:parquet-avro:jar:1.8.2:compile
[INFO] |  +- org.apache.avro:avro-mapred:jar:hadoop2:1.8.1:provided
[INFO] |  |  +- org.apache.avro:avro-ipc:jar:1.8.1:provided
[INFO] |  \- org.apache.parquet:parquet-avro:jar:1.8.2:compile
[INFO] |  \- org.apache.avro:avro:jar:1.8.1:compile
[INFO] |  |  \- org.apache.parquet:parquet-avro:jar:1.8.2:compile
[INFO] |  |  \- org.apache.avro:avro:jar:1.8.1:compile

(duplicates here because I am running from the parent directory)

Avro 1.7.7 is present in the Spark lib folder

$ ls /usr/local/Cellar/apache-spark/2.1.0/libexec/jars/avro-*
/usr/local/Cellar/apache-spark/2.1.0/libexec/jars/avro-1.7.7.jar
/usr/local/Cellar/apache-spark/2.1.0/libexec/jars/avro-ipc-1.7.7.jar
/usr/local/Cellar/apache-spark/2.1.0/libexec/jars/avro-mapred-1.7.7-hadoop2.jar

Are we not able to fully override these at runtime?

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jan 24, 2017

Member

Sigh! How does this work right now? I show us on Avro 1.8.0 on ToT?

Member

fnothaft commented Jan 24, 2017

Sigh! How does this work right now? I show us on Avro 1.8.0 on ToT?

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Jan 24, 2017

Member

No. As the parquet 1.8.2 release is still being voted on, we have the chance to -1 if it is at fault.

Member

heuermh commented Jan 24, 2017

No. As the parquet 1.8.2 release is still being voted on, we have the chance to -1 if it is at fault.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Feb 21, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1810/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains b463c5e # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision b463c5e (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f b463c5e5d0b076f54a213ddf3b59d2532afd61f7First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Feb 21, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1810/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains b463c5e # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision b463c5e (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f b463c5e5d0b076f54a213ddf3b59d2532afd61f7First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Feb 21, 2017

Member

Am I seeing this right, are only the Scala 2.10 builds failing now?

Whenever I run scripts/jenkins-test locally, it fails with

+ echo 'Applying move_to_xyz script marred a pom.xml file.'
Applying move_to_xyz script marred a pom.xml file.
+ echo Exiting...
Exiting...
+ exit 1
Member

heuermh commented Feb 21, 2017

Am I seeing this right, are only the Scala 2.10 builds failing now?

Whenever I run scripts/jenkins-test locally, it fails with

+ echo 'Applying move_to_xyz script marred a pom.xml file.'
Applying move_to_xyz script marred a pom.xml file.
+ echo Exiting...
Exiting...
+ exit 1
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Feb 21, 2017

Member

Yes, although your problems are worse than you imagined. See, only the Scala 2.10 builds run the integration tests. Check out your stacktrace from the integration tests:

2017-02-21 13:27:20 ERROR Executor:95 - Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.NoSuchMethodError: org.apache.avro.Schema.getLogicalType()Lorg/apache/avro/LogicalType;
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:178)
	at org.apache.parquet.avro.AvroSchemaConverter.convertUnion(AvroSchemaConverter.java:214)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:171)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:130)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:227)
	at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:124)
	at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:115)
	at org.apache.parquet.avro.AvroWriteSupport.init(AvroWriteSupport.java:117)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:288)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:262)
	at org.apache.spark.rdd.InstrumentedOutputFormat.getRecordWriter(InstrumentedOutputFormat.scala:35)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1107)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1091)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
	at org.apache.spark.scheduler.Task.run(Task.scala:89)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Looks like a Parquet/Avro version incompatibility.

Member

fnothaft commented Feb 21, 2017

Yes, although your problems are worse than you imagined. See, only the Scala 2.10 builds run the integration tests. Check out your stacktrace from the integration tests:

2017-02-21 13:27:20 ERROR Executor:95 - Exception in task 0.0 in stage 0.0 (TID 0)
java.lang.NoSuchMethodError: org.apache.avro.Schema.getLogicalType()Lorg/apache/avro/LogicalType;
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:178)
	at org.apache.parquet.avro.AvroSchemaConverter.convertUnion(AvroSchemaConverter.java:214)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:171)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:130)
	at org.apache.parquet.avro.AvroSchemaConverter.convertField(AvroSchemaConverter.java:227)
	at org.apache.parquet.avro.AvroSchemaConverter.convertFields(AvroSchemaConverter.java:124)
	at org.apache.parquet.avro.AvroSchemaConverter.convert(AvroSchemaConverter.java:115)
	at org.apache.parquet.avro.AvroWriteSupport.init(AvroWriteSupport.java:117)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:288)
	at org.apache.parquet.hadoop.ParquetOutputFormat.getRecordWriter(ParquetOutputFormat.java:262)
	at org.apache.spark.rdd.InstrumentedOutputFormat.getRecordWriter(InstrumentedOutputFormat.scala:35)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1107)
	at org.apache.spark.rdd.PairRDDFunctions$$anonfun$saveAsNewAPIHadoopDataset$1$$anonfun$12.apply(PairRDDFunctions.scala:1091)
	at org.apache.spark.scheduler.ResultTask.runTask(ResultTask.scala:66)
	at org.apache.spark.scheduler.Task.run(Task.scala:89)
	at org.apache.spark.executor.Executor$TaskRunner.run(Executor.scala:214)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

Looks like a Parquet/Avro version incompatibility.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Feb 21, 2017

Member

That is the problem I'm trying to solve. You asked earlier

How does this work right now?

I don't know how it works right now, nor do I understand where the < 1.8.0 version of avro is coming into play.

Member

heuermh commented Feb 21, 2017

That is the problem I'm trying to solve. You asked earlier

How does this work right now?

I don't know how it works right now, nor do I understand where the < 1.8.0 version of avro is coming into play.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Feb 21, 2017

Member

Oh hah, right. Yeah, let me circle back and look at this.

Member

fnothaft commented Feb 21, 2017

Oh hah, right. Yeah, let me circle back and look at this.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Feb 22, 2017

Member

Reported as Apache Spark JIRA issue
https://issues.apache.org/jira/browse/SPARK-19697

Member

heuermh commented Feb 22, 2017

Reported as Apache Spark JIRA issue
https://issues.apache.org/jira/browse/SPARK-19697

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Feb 22, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1814/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains d603da6 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision d603da6 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f d603da6ff3078c433ac94bac119a3a456156abdaFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Feb 22, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1814/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains d603da6 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision d603da6 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f d603da6ff3078c433ac94bac119a3a456156abdaFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 3, 2017

Coverage Status

Changes Unknown when pulling 9f10a85 on heuermh:dep-versions into ** on bigdatagenomics:master**.

coveralls commented Mar 3, 2017

Coverage Status

Changes Unknown when pulling 9f10a85 on heuermh:dep-versions into ** on bigdatagenomics:master**.

1 similar comment
@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 3, 2017

Coverage Status

Changes Unknown when pulling 9f10a85 on heuermh:dep-versions into ** on bigdatagenomics:master**.

coveralls commented Mar 3, 2017

Coverage Status

Changes Unknown when pulling 9f10a85 on heuermh:dep-versions into ** on bigdatagenomics:master**.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 3, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1833/
Test PASSed.

AmplabJenkins commented Mar 3, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1833/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 14, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1873/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 398b9b13772f4e6e59307ca2d749a9e628513892 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 398b9b13772f4e6e59307ca2d749a9e628513892 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 398b9b13772f4e6e59307ca2d749a9e628513892First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Mar 14, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1873/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 398b9b13772f4e6e59307ca2d749a9e628513892 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 398b9b13772f4e6e59307ca2d749a9e628513892 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 398b9b13772f4e6e59307ca2d749a9e628513892First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Mar 14, 2017

Member

How about we do a shakedown on the cluster before merging? Anywho, looks like the 0.2.12 update brought in some unittest failures?

Member

fnothaft commented Mar 14, 2017

How about we do a shakedown on the cluster before merging? Anywho, looks like the 0.2.12 update brought in some unittest failures?

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Mar 14, 2017

Member

Also, any chance I can get you to rebase this?

Member

fnothaft commented Mar 14, 2017

Also, any chance I can get you to rebase this?

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 14, 2017

Member

Yep, looks like a rebase should help. Unit tests work fine for me locally.

Member

heuermh commented Mar 14, 2017

Yep, looks like a rebase should help. Unit tests work fine for me locally.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 14, 2017

Coverage Status

Coverage remained the same at 76.659% when pulling 812816b on heuermh:dep-versions into b8477dc on bigdatagenomics:master.

coveralls commented Mar 14, 2017

Coverage Status

Coverage remained the same at 76.659% when pulling 812816b on heuermh:dep-versions into b8477dc on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 14, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1875/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 3b93d21 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 3b93d21 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3b93d21628948f35070a7713fe2cc8f8e81ec537First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Mar 14, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1875/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 3b93d21 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 3b93d21 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 3b93d21628948f35070a7713fe2cc8f8e81ec537First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 15, 2017

Member

I am not seeing these unit test failures locally. Any ideas? I'll try scripts/jenkins-test locally next.

Member

heuermh commented Mar 15, 2017

I am not seeing these unit test failures locally. Any ideas? I'll try scripts/jenkins-test locally next.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 15, 2017

Member

Not sure what the coveralls token should be...

$ HADOOP_VERSION=2.6.0 SPARK_VERSION=1.6.1 ./scripts/jenkins-test
...
# if this is a pull request, we need to set the coveralls pr id
if [[ ! -z $ghprbPullId ]];
then
    COVERALLS_PRB_OPTION="-DpullRequest=${ghprbPullId}"
fi
+ [[ ! -z '' ]]

# coveralls token should not be visible
set +x +v
+ set +x +v
Coveralls token is not set. Exiting...
Member

heuermh commented Mar 15, 2017

Not sure what the coveralls token should be...

$ HADOOP_VERSION=2.6.0 SPARK_VERSION=1.6.1 ./scripts/jenkins-test
...
# if this is a pull request, we need to set the coveralls pr id
if [[ ! -z $ghprbPullId ]];
then
    COVERALLS_PRB_OPTION="-DpullRequest=${ghprbPullId}"
fi
+ [[ ! -z '' ]]

# coveralls token should not be visible
set +x +v
+ set +x +v
Coveralls token is not set. Exiting...
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Mar 15, 2017

Member

@heuermh can you back out the upgrade to utils 0.2.12? I'm wondering if we scoobied something in the IntervalTree code in the 0.2.12 release.

Member

fnothaft commented Mar 15, 2017

@heuermh can you back out the upgrade to utils 0.2.12? I'm wondering if we scoobied something in the IntervalTree code in the 0.2.12 release.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Mar 16, 2017

Coverage Status

Coverage remained the same at 76.479% when pulling 40f774c on heuermh:dep-versions into cf39e6c on bigdatagenomics:master.

coveralls commented Mar 16, 2017

Coverage Status

Coverage remained the same at 76.479% when pulling 40f774c on heuermh:dep-versions into cf39e6c on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 16, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1882/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains bbc8a00 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision bbc8a00 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f bbc8a00d637fafa01e88312da46ebfd2f49747dbFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Mar 16, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1882/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains bbc8a00 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision bbc8a00 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f bbc8a00d637fafa01e88312da46ebfd2f49747dbFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 16, 2017

Member

I still can't reproduce these failures locally, and the failures on Jenkins are not consistent.

Member

heuermh commented Mar 16, 2017

I still can't reproduce these failures locally, and the failures on Jenkins are not consistent.

@devin-petersohn

This comment has been minimized.

Show comment
Hide comment
@devin-petersohn

devin-petersohn Mar 16, 2017

Member

I found the issue. It revolves around our thread unsafe variable that was meant to speed up the queries in IntervalArray. I am having the same issue in #1324. I think we should revert that change immediately. I am working on a PR for utils. Here are the lines in question: Thread unsafe variable declaration, Use in the get method. Once I took this out I no longer got the failures.

Member

devin-petersohn commented Mar 16, 2017

I found the issue. It revolves around our thread unsafe variable that was meant to speed up the queries in IntervalArray. I am having the same issue in #1324. I think we should revert that change immediately. I am working on a PR for utils. Here are the lines in question: Thread unsafe variable declaration, Use in the get method. Once I took this out I no longer got the failures.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 6, 2017

Coverage Status

Coverage remained the same at 81.535% when pulling 9abb159 on heuermh:dep-versions into 98d2c92 on bigdatagenomics:master.

coveralls commented Apr 6, 2017

Coverage Status

Coverage remained the same at 81.535% when pulling 9abb159 on heuermh:dep-versions into 98d2c92 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 6, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1933/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 25a0a3fec82893638593d8d849a6cee0d6a99086 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 25a0a3fec82893638593d8d849a6cee0d6a99086 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 25a0a3fec82893638593d8d849a6cee0d6a99086First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Apr 6, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1933/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1360/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 25a0a3fec82893638593d8d849a6cee0d6a99086 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1360/merge^{commit} # timeout=10Checking out Revision 25a0a3fec82893638593d8d849a6cee0d6a99086 (origin/pr/1360/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 25a0a3fec82893638593d8d849a6cee0d6a99086First time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 6, 2017

Member

Fixes #1300

Member

heuermh commented Apr 6, 2017

Fixes #1300

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Apr 6, 2017

Coverage Status

Coverage remained the same at 81.535% when pulling 98ba92e on heuermh:dep-versions into 98d2c92 on bigdatagenomics:master.

coveralls commented Apr 6, 2017

Coverage Status

Coverage remained the same at 81.535% when pulling 98ba92e on heuermh:dep-versions into 98d2c92 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 6, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1934/
Test PASSed.

AmplabJenkins commented Apr 6, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1934/
Test PASSed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 6, 2017

Member

@heuermh this is good to go now, correct?

Member

fnothaft commented Apr 6, 2017

@heuermh this is good to go now, correct?

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 6, 2017

Member

Yes, I believe so

Member

heuermh commented Apr 6, 2017

Yes, I believe so

@fnothaft fnothaft merged commit 9505d47 into bigdatagenomics:master Apr 6, 2017

3 checks passed

codacy/pr Good work! A positive pull request.
Details
coverage/coveralls Coverage remained the same at 81.535%
Details
default Merged build finished.
Details
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 6, 2017

Member

Merged! Thanks @heuermh!

Member

fnothaft commented Apr 6, 2017

Merged! Thanks @heuermh!

@heuermh heuermh deleted the heuermh:dep-versions branch Apr 6, 2017

@@ -93,14 +93,17 @@
<dependency>
<groupId>org.scala-lang</groupId>
<artifactId>scala-library</artifactId>
<scope>compile</scope>

This comment has been minimized.

@ryan-williams

ryan-williams Apr 6, 2017

Member

why add all these, OOC?

@ryan-williams

ryan-williams Apr 6, 2017

Member

why add all these, OOC?

This comment has been minimized.

@heuermh

heuermh Apr 6, 2017

Member

for doc mostly, scope is compile by default; these make it explicit

@heuermh

heuermh Apr 6, 2017

Member

for doc mostly, scope is compile by default; these make it explicit

@heuermh heuermh moved this from Triage to Completed in Release 0.23.0 May 30, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment