New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAM-528][ADAM-533] Adds new RegionJoin impl that is shuffle-based #534

Merged
merged 1 commit into from Jan 9, 2015

Conversation

Projects
None yet
3 participants
@laserson
Contributor

laserson commented Dec 24, 2014

First pass shuffle-based join impl. Passes some rudimentary tests.

Certainly needs docs and more tests.

Fixes #528.
Fixes #533.

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Dec 24, 2014

Contributor

@tdanford @fnothaft ping for review.

Contributor

laserson commented Dec 24, 2014

@tdanford @fnothaft ping for review.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Dec 24, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/514/

Build result: FAILURE

GitHub pull request #534 of commit 57a7463 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/534/merge^{commit} # timeout=10Checking out Revision 3af839a (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 3af839a > git rev-list b409bd0004a00c2dac712b5c567963b3e013c8e3 # timeout=10Triggering ADAM-prb ? 2.3.0,centosTriggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 1.0.4,centosADAM-prb ? 2.3.0,centos completed with result FAILUREADAM-prb ? 2.2.0,centos completed with result FAILUREADAM-prb ? 1.0.4,centos completed with result FAILURE
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/514/

Build result: FAILURE

GitHub pull request #534 of commit 57a7463 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/534/merge^{commit} # timeout=10Checking out Revision 3af839a (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 3af839a > git rev-list b409bd0004a00c2dac712b5c567963b3e013c8e3 # timeout=10Triggering ADAM-prb ? 2.3.0,centosTriggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 1.0.4,centosADAM-prb ? 2.3.0,centos completed with result FAILUREADAM-prb ? 2.2.0,centos completed with result FAILUREADAM-prb ? 1.0.4,centos completed with result FAILURE
Test FAILed.

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Dec 24, 2014

Contributor

oops, my repo is in a dirty state. fix momentarily

Contributor

laserson commented Dec 24, 2014

oops, my repo is in a dirty state. fix momentarily

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Dec 24, 2014

Member

@laserson will look tommorrow AM.

Member

fnothaft commented Dec 24, 2014

@laserson will look tommorrow AM.

@laserson laserson changed the title from Adds new RegionJoin impl that is shuffle-based to [ADAM-528][ADAM-533] Adds new RegionJoin impl that is shuffle-based Dec 24, 2014

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Dec 24, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/515/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/515/
Test PASSed.

Show outdated Hide outdated adam-core/src/main/scala/org/bdgenomics/adam/rdd/RegionJoin2.scala
import scala.math._
import scala.reflect.ClassTag
object RegionJoin2 {

This comment has been minimized.

@fnothaft

fnothaft Dec 24, 2014

Member

Maybe we should change the names of the respective joins to BroadcastRegionJoin and ShuffleRegionJoin?

@fnothaft

fnothaft Dec 24, 2014

Member

Maybe we should change the names of the respective joins to BroadcastRegionJoin and ShuffleRegionJoin?

This comment has been minimized.

@laserson

laserson Dec 24, 2014

Contributor

+1

@laserson

laserson Dec 24, 2014

Contributor

+1

Show outdated Hide outdated adam-core/src/main/scala/org/bdgenomics/adam/rdd/RegionJoin2.scala
}
}
case class SortedIntervalPartitionJoin[T, U](region: ReferenceRegion, leftIter: Iterator[((ReferenceRegion, Int), T)], rightIter: Iterator[((ReferenceRegion, Int), U)])

This comment has been minimized.

@fnothaft

fnothaft Dec 24, 2014

Member

Just making sure I understand this part correctly: SortedIntervalPartitionJoin takes a zipped partition (i.e, the leftIter and the rightIter) and internally does the sort merge join?

@fnothaft

fnothaft Dec 24, 2014

Member

Just making sure I understand this part correctly: SortedIntervalPartitionJoin takes a zipped partition (i.e, the leftIter and the rightIter) and internally does the sort merge join?

This comment has been minimized.

@laserson

laserson Dec 24, 2014

Contributor

That's correct.

@laserson

laserson Dec 24, 2014

Contributor

That's correct.

Show outdated Hide outdated adam-core/src/main/scala/org/bdgenomics/adam/rdd/RegionJoin2.scala
case class SortedIntervalPartitionJoin[T, U](region: ReferenceRegion, leftIter: Iterator[((ReferenceRegion, Int), T)], rightIter: Iterator[((ReferenceRegion, Int), U)])
extends Iterator[(T, U)] {
// inspired by bedtools2 chromsweep
private val left: BufferedIterator[((ReferenceRegion, Int), T)] = leftIter.buffered

This comment has been minimized.

@fnothaft

fnothaft Dec 24, 2014

Member

Out of curiosity, why use a BufferedIterator here?

@fnothaft

fnothaft Dec 24, 2014

Member

Out of curiosity, why use a BufferedIterator here?

This comment has been minimized.

@laserson

laserson Dec 24, 2014

Contributor

Seemed easier to implement the logic with the ability to peek at the first element. Is there a reason to avoid BufferedIterator?

@laserson

laserson Dec 24, 2014

Contributor

Seemed easier to implement the logic with the ability to peek at the first element. Is there a reason to avoid BufferedIterator?

Show outdated Hide outdated adam-core/src/main/scala/org/bdgenomics/adam/rdd/RegionJoin2.scala
private def pruneCache(to: Long): Unit = {
while (cache.head._1.end <= to) {
cache.trimStart(1)

This comment has been minimized.

@fnothaft

fnothaft Dec 24, 2014

Member

Couldn't this more efficiently be written as a trimWhile or dropWhile?

@fnothaft

fnothaft Dec 24, 2014

Member

Couldn't this more efficiently be written as a trimWhile or dropWhile?

This comment has been minimized.

@laserson

laserson Dec 24, 2014

Contributor

Yes, as long as dropWhile actually mutates cache, which I think it does.

@laserson

laserson Dec 24, 2014

Contributor

Yes, as long as dropWhile actually mutates cache, which I think it does.

Show outdated Hide outdated adam-core/src/main/scala/org/bdgenomics/adam/rdd/RegionJoin2.scala
private val cache: ListBuffer[(ReferenceRegion, U)] = ListBuffer.empty
private var prevLeftRegion: ReferenceRegion = _
private def advanceCache(until: Long): Unit = {

This comment has been minimized.

@fnothaft

fnothaft Dec 24, 2014

Member

So this is taking the values from the right iterator and adding them to the cache if they start at/before the until position, correct?

@fnothaft

fnothaft Dec 24, 2014

Member

So this is taking the values from the right iterator and adding them to the cache if they start at/before the until position, correct?

This comment has been minimized.

@laserson

laserson Dec 24, 2014

Contributor

Correct.

@laserson

laserson Dec 24, 2014

Contributor

Correct.

Show outdated Hide outdated adam-core/src/main/scala/org/bdgenomics/adam/rdd/RegionJoin2.scala
private def advanceCache(until: Long): Unit = {
while (right.hasNext && right.head._1._1.start < until) {
val x = right.next()

This comment has been minimized.

@fnothaft

fnothaft Dec 24, 2014

Member

Doesn't this drop the current head of right? E.g., I'll call x = right.next until I get to the head of right that is >= until. So, that x >= until never gets added to the cache, no?

@fnothaft

fnothaft Dec 24, 2014

Member

Doesn't this drop the current head of right? E.g., I'll call x = right.next until I get to the head of right that is >= until. So, that x >= until never gets added to the cache, no?

This comment has been minimized.

@laserson

laserson Dec 24, 2014

Contributor

I'm not sure what you mean. I look at the top of the right iterator. If it starts to the left of until, I pull it out of the iterator and add it to the cache. Now head points to the next guy. If he starts to the right of until, I stop there.

@laserson

laserson Dec 24, 2014

Contributor

I'm not sure what you mean. I look at the top of the right iterator. If it starts to the left of until, I pull it out of the iterator and add it to the cache. Now head points to the next guy. If he starts to the right of until, I stop there.

Show outdated Hide outdated adam-core/src/main/scala/org/bdgenomics/adam/rdd/RegionJoin2.scala
val ((nextLeftRegion, _), nextLeft) = left.next
// ...so check whether I need to advance the cache
// (where nextLeftRegion's end is further than prevLeftRegion's end)...
if (prevLeftRegion == null || nextLeftRegion.end > prevLeftRegion.end) {

This comment has been minimized.

@fnothaft

fnothaft Dec 24, 2014

Member

Why do you need the null checks here?

@fnothaft

fnothaft Dec 24, 2014

Member

Why do you need the null checks here?

This comment has been minimized.

@laserson

laserson Dec 24, 2014

Contributor

For initialization. On the first pass, prevLeftRegion hasn't been initialized, so I force it to execute advanceCache.

@laserson

laserson Dec 24, 2014

Contributor

For initialization. On the first pass, prevLeftRegion hasn't been initialized, so I force it to execute advanceCache.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Dec 24, 2014

Member

Nice! Looks like a pretty cool implementation here. I dropped a few questions inline, but otherwise LGTM.

Member

fnothaft commented Dec 24, 2014

Nice! Looks like a pretty cool implementation here. I dropped a few questions inline, but otherwise LGTM.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Dec 26, 2014

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/517/

Build result: ABORTED

GitHub pull request #534 of commit 1f1f659 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/534/merge^{commit} # timeout=10Checking out Revision 6044286 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 6044286 > git rev-list b409bd0004a00c2dac712b5c567963b3e013c8e3 # timeout=10Triggering ADAM-prb ? 2.3.0,centosTriggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 1.0.4,centosADAM-prb ? 2.3.0,centos completed with result SUCCESSADAM-prb ? 2.2.0,centos completed with result FAILUREADAM-prb ? 1.0.4,centos completed with result ABORTED
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/517/

Build result: ABORTED

GitHub pull request #534 of commit 1f1f659 automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/534/merge^{commit} # timeout=10Checking out Revision 6044286 (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 6044286 > git rev-list b409bd0004a00c2dac712b5c567963b3e013c8e3 # timeout=10Triggering ADAM-prb ? 2.3.0,centosTriggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 1.0.4,centosADAM-prb ? 2.3.0,centos completed with result SUCCESSADAM-prb ? 2.2.0,centos completed with result FAILUREADAM-prb ? 1.0.4,centos completed with result ABORTED
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 5, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/518/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/518/
Test PASSed.

Uri Laserson
[ADAM-528][ADAM-533] Adds new RegionJoin impl that is shuffle-based
Also renames GenomicRegionPartitioner to GenomicPositionPartitioner

Fixes #528.
Fixes #533.
@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Jan 6, 2015

Contributor

@tdanford @fnothaft What is necessary to move this forward? I added scaladocs, and there are some tests, though there could be more. One issues is I made a few changes that I thought would be relatively cosmetic, and now the build hangs with some Akka problem. Any idea what's going on here?

2015-01-05 18:36:27 ERROR OneForOneStrategy:66 - Actor not found for: ActorSelection[Anchor(akka.tcp://sparkDriver@localhost:7077/), Path(/user/HeartbeatReceiver)]
akka.actor.ActorInitializationException: exception during creation
    at akka.actor.ActorInitializationException$.apply(Actor.scala:164)
    at akka.actor.ActorCell.create(ActorCell.scala:596)
    at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
    at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
    at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
    at akka.dispatch.Mailbox.run(Mailbox.scala:219)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://sparkDriver@localhost:7077/), Path(/user/HeartbeatReceiver)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:65)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:63)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
    at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
    at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
    at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
    at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:110)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
    at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:267)
    at akka.actor.EmptyLocalActorRef.specialHandle(ActorRef.scala:508)
    at akka.actor.DeadLetterActorRef.specialHandle(ActorRef.scala:541)
    at akka.actor.DeadLetterActorRef.$bang(ActorRef.scala:531)
    at akka.remote.RemoteActorRefProvider$RemoteDeadLetterActorRef.$bang(RemoteActorRefProvider.scala:87)
    at akka.remote.EndpointWriter.postStop(Endpoint.scala:561)
    at akka.actor.Actor$class.aroundPostStop(Actor.scala:475)
    at akka.remote.EndpointActor.aroundPostStop(Endpoint.scala:415)
    at akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:210)
    at akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172)
    at akka.actor.ActorCell.terminate(ActorCell.scala:369)
    at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462)
    ... 8 more
Contributor

laserson commented Jan 6, 2015

@tdanford @fnothaft What is necessary to move this forward? I added scaladocs, and there are some tests, though there could be more. One issues is I made a few changes that I thought would be relatively cosmetic, and now the build hangs with some Akka problem. Any idea what's going on here?

2015-01-05 18:36:27 ERROR OneForOneStrategy:66 - Actor not found for: ActorSelection[Anchor(akka.tcp://sparkDriver@localhost:7077/), Path(/user/HeartbeatReceiver)]
akka.actor.ActorInitializationException: exception during creation
    at akka.actor.ActorInitializationException$.apply(Actor.scala:164)
    at akka.actor.ActorCell.create(ActorCell.scala:596)
    at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:456)
    at akka.actor.ActorCell.systemInvoke(ActorCell.scala:478)
    at akka.dispatch.Mailbox.processAllSystemMessages(Mailbox.scala:263)
    at akka.dispatch.Mailbox.run(Mailbox.scala:219)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(AbstractDispatcher.scala:393)
    at scala.concurrent.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at scala.concurrent.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at scala.concurrent.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at scala.concurrent.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
Caused by: akka.actor.ActorNotFound: Actor not found for: ActorSelection[Anchor(akka.tcp://sparkDriver@localhost:7077/), Path(/user/HeartbeatReceiver)]
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:65)
    at akka.actor.ActorSelection$$anonfun$resolveOne$1.apply(ActorSelection.scala:63)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:32)
    at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.processBatch$1(BatchingExecutor.scala:67)
    at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply$mcV$sp(BatchingExecutor.scala:82)
    at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
    at akka.dispatch.BatchingExecutor$Batch$$anonfun$run$1.apply(BatchingExecutor.scala:59)
    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:72)
    at akka.dispatch.BatchingExecutor$Batch.run(BatchingExecutor.scala:58)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.unbatchedExecute(Future.scala:74)
    at akka.dispatch.BatchingExecutor$class.execute(BatchingExecutor.scala:110)
    at akka.dispatch.ExecutionContexts$sameThreadExecutionContext$.execute(Future.scala:73)
    at scala.concurrent.impl.CallbackRunnable.executeWithValue(Promise.scala:40)
    at scala.concurrent.impl.Promise$DefaultPromise.tryComplete(Promise.scala:248)
    at akka.pattern.PromiseActorRef.$bang(AskSupport.scala:267)
    at akka.actor.EmptyLocalActorRef.specialHandle(ActorRef.scala:508)
    at akka.actor.DeadLetterActorRef.specialHandle(ActorRef.scala:541)
    at akka.actor.DeadLetterActorRef.$bang(ActorRef.scala:531)
    at akka.remote.RemoteActorRefProvider$RemoteDeadLetterActorRef.$bang(RemoteActorRefProvider.scala:87)
    at akka.remote.EndpointWriter.postStop(Endpoint.scala:561)
    at akka.actor.Actor$class.aroundPostStop(Actor.scala:475)
    at akka.remote.EndpointActor.aroundPostStop(Endpoint.scala:415)
    at akka.actor.dungeon.FaultHandling$class.akka$actor$dungeon$FaultHandling$$finishTerminate(FaultHandling.scala:210)
    at akka.actor.dungeon.FaultHandling$class.terminate(FaultHandling.scala:172)
    at akka.actor.ActorCell.terminate(ActorCell.scala:369)
    at akka.actor.ActorCell.invokeAll$1(ActorCell.scala:462)
    ... 8 more
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jan 6, 2015

Member

I'm good with merging once the tests pass; I can give the test issue a look tomorrow.

Member

fnothaft commented Jan 6, 2015

I'm good with merging once the tests pass; I can give the test issue a look tomorrow.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 6, 2015

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/519/

Build result: ABORTED

GitHub pull request #534 of commit 85b17fc automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/534/merge^{commit} # timeout=10Checking out Revision 50ed66e (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 50ed66e > git rev-list b409bd0004a00c2dac712b5c567963b3e013c8e3 # timeout=10Triggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 1.0.4,centosTriggering ADAM-prb ? 2.3.0,centosADAM-prb ? 2.2.0,centos completed with result SUCCESSADAM-prb ? 1.0.4,centos completed with result ABORTEDADAM-prb ? 2.3.0,centos completed with result ABORTED
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/519/

Build result: ABORTED

GitHub pull request #534 of commit 85b17fc automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/534/merge^{commit} # timeout=10Checking out Revision 50ed66e (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 50ed66e > git rev-list b409bd0004a00c2dac712b5c567963b3e013c8e3 # timeout=10Triggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 1.0.4,centosTriggering ADAM-prb ? 2.3.0,centosADAM-prb ? 2.2.0,centos completed with result SUCCESSADAM-prb ? 1.0.4,centos completed with result ABORTEDADAM-prb ? 2.3.0,centos completed with result ABORTED
Test FAILed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jan 6, 2015

Member

Jenkins, retest this please.

Member

fnothaft commented Jan 6, 2015

Jenkins, retest this please.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 6, 2015

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/520/

Build result: FAILURE

GitHub pull request #534 of commit 85b17fc automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/534/merge^{commit} # timeout=10Checking out Revision 50ed66e (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 50ed66e > git rev-list b409bd0004a00c2dac712b5c567963b3e013c8e3 # timeout=10Triggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 1.0.4,centosTriggering ADAM-prb ? 2.3.0,centosADAM-prb ? 2.2.0,centos completed with result SUCCESSADAM-prb ? 1.0.4,centos completed with result SUCCESSADAM-prb ? 2.3.0,centos completed with result FAILURE
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/520/

Build result: FAILURE

GitHub pull request #534 of commit 85b17fc automatically merged.[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/534/merge^{commit} # timeout=10Checking out Revision 50ed66e (detached) > git config core.sparsecheckout # timeout=10 > git checkout -f 50ed66e > git rev-list b409bd0004a00c2dac712b5c567963b3e013c8e3 # timeout=10Triggering ADAM-prb ? 2.2.0,centosTriggering ADAM-prb ? 1.0.4,centosTriggering ADAM-prb ? 2.3.0,centosADAM-prb ? 2.2.0,centos completed with result SUCCESSADAM-prb ? 1.0.4,centos completed with result SUCCESSADAM-prb ? 2.3.0,centos completed with result FAILURE
Test FAILed.

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Jan 6, 2015

Contributor

Jenkins, retest this please.

Contributor

laserson commented Jan 6, 2015

Jenkins, retest this please.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jan 6, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/521/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/521/
Test PASSed.

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Jan 6, 2015

Contributor

@fnothaft well there you go

Contributor

laserson commented Jan 6, 2015

@fnothaft well there you go

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Jan 6, 2015

Contributor

@fnothaft @tdanford ok, please review, and let me know if there are some other things to do before merging :)

Contributor

laserson commented Jan 6, 2015

@fnothaft @tdanford ok, please review, and let me know if there are some other things to do before merging :)

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jan 6, 2015

Member

I am +1 to merge.

Member

fnothaft commented Jan 6, 2015

I am +1 to merge.

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Jan 9, 2015

Contributor

@fnothaft Is it uncouth to merge my own PR after it's +1'd?

Contributor

laserson commented Jan 9, 2015

@fnothaft Is it uncouth to merge my own PR after it's +1'd?

fnothaft added a commit that referenced this pull request Jan 9, 2015

Merge pull request #534 from laserson/ADAM-528
[ADAM-528][ADAM-533] Adds new RegionJoin impl that is shuffle-based

@fnothaft fnothaft merged commit 2cd4fdb into bigdatagenomics:master Jan 9, 2015

1 check passed

default Merged build finished.
Details
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jan 9, 2015

Member

@laserson resolved that problem for you ;). Thanks for the contribution!

Member

fnothaft commented Jan 9, 2015

@laserson resolved that problem for you ;). Thanks for the contribution!

@laserson laserson deleted the laserson:ADAM-528 branch Jan 10, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment