add function:filterByOverlappingRegions #1287

Merged
merged 2 commits into from Nov 23, 2016

Conversation

Projects
None yet
4 participants
@liamlee
Contributor

liamlee commented Nov 21, 2016

for multiple regions data select.

liming
add function:filterByOverlappingRegions
for multiple regions data select.
@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 21, 2016

Can one of the admins verify this patch?

Can one of the admins verify this patch?

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Nov 21, 2016

Member

Jenkins, test this please

Member

heuermh commented Nov 21, 2016

Jenkins, test this please

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 21, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1634/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1287/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 57397933361bd4535c372f6c54dfa496a16372a2 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1287/merge^{commit} # timeout=10Checking out Revision 57397933361bd4535c372f6c54dfa496a16372a2 (origin/pr/1287/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 57397933361bd4535c372f6c54dfa496a16372a2First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1634/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1287/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 57397933361bd4535c372f6c54dfa496a16372a2 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1287/merge^{commit} # timeout=10Checking out Revision 57397933361bd4535c372f6c54dfa496a16372a2 (origin/pr/1287/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 57397933361bd4535c372f6c54dfa496a16372a2First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Nov 21, 2016

Member

From the failed Jenkins build

[INFO] Compiling 71 source files to /home/jenkins/workspace/ADAM-prb/HADOOP_VERSION/2.6.0/SCALAVER/2.10/SPARK_VERSION/1.5.2/label/centos/adam-core/target/2.10.6/test-classes at 1479736143868
[ERROR] /home/jenkins/workspace/ADAM-prb/HADOOP_VERSION/2.6.0/SCALAVER/2.10/SPARK_VERSION/1.5.2/label/centos/adam-core/src/test/scala/org/bdgenomics/adam/rdd/read/AlignmentRecordRDDSuite.scala:110: error: not found: value resourcePath
[ERROR]     val inputPath = resourcePath("artificial.sam")
[ERROR]                     ^
[ERROR] one error found
Member

heuermh commented Nov 21, 2016

From the failed Jenkins build

[INFO] Compiling 71 source files to /home/jenkins/workspace/ADAM-prb/HADOOP_VERSION/2.6.0/SCALAVER/2.10/SPARK_VERSION/1.5.2/label/centos/adam-core/target/2.10.6/test-classes at 1479736143868
[ERROR] /home/jenkins/workspace/ADAM-prb/HADOOP_VERSION/2.6.0/SCALAVER/2.10/SPARK_VERSION/1.5.2/label/centos/adam-core/src/test/scala/org/bdgenomics/adam/rdd/read/AlignmentRecordRDDSuite.scala:110: error: not found: value resourcePath
[ERROR]     val inputPath = resourcePath("artificial.sam")
[ERROR]                     ^
[ERROR] one error found
@fnothaft

Only two small changes. Otherwise looks very good! Thank you for the contribution.

+
+ querys.map(query => {
+ regions.exists(_.overlaps(query))
+ }).reduce((a, b) => a || b)

This comment has been minimized.

@fnothaft

fnothaft Nov 21, 2016

Member

I would replace reduce((a, b) => a || b) with fold(false)((a, b) => a || b) as reduce will fail on an empty collection. You can get an empty result from getReferenceRegions(elem) with unmapped reads.

@fnothaft

fnothaft Nov 21, 2016

Member

I would replace reduce((a, b) => a || b) with fold(false)((a, b) => a || b) as reduce will fail on an empty collection. You can get an empty result from getReferenceRegions(elem) with unmapped reads.

This comment has been minimized.

@liamlee

liamlee Nov 22, 2016

Contributor

This is a very helpful advise, Thanks!

@liamlee

liamlee Nov 22, 2016

Contributor

This is a very helpful advise, Thanks!

@@ -106,6 +106,16 @@ class AlignmentRecordRDDSuite extends ADAMFunSuite {
assert(coverage.rdd.filter(r => r.start == 30).first.count == pointCoverage)
}
+ sparkTest("test filterByOverlappingRegions") {
+ val inputPath = resourcePath("artificial.sam")

This comment has been minimized.

@fnothaft

fnothaft Nov 21, 2016

Member

You'll want to change resourcePath("artificial.sam") to testFile("artificial.sam") and then this should compile.

@fnothaft

fnothaft Nov 21, 2016

Member

You'll want to change resourcePath("artificial.sam") to testFile("artificial.sam") and then this should compile.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 22, 2016

Member

Jenkins, test this please.

Member

fnothaft commented Nov 22, 2016

Jenkins, test this please.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 22, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1635/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1635/
Test PASSed.

@fnothaft fnothaft merged commit 9a6ceb3 into bigdatagenomics:master Nov 23, 2016

1 check passed

default Merged build finished.
Details
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 23, 2016

Member

Merged! Thank you very much @liamlee for the contribution!

Member

fnothaft commented Nov 23, 2016

Merged! Thank you very much @liamlee for the contribution!

@liamlee

This comment has been minimized.

Show comment
Hide comment
@liamlee

liamlee Nov 23, 2016

Contributor

My pleasure.

Contributor

liamlee commented Nov 23, 2016

My pleasure.

@liamlee liamlee deleted the liamlee:feature branch Nov 23, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment