New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAM-1889][ADAM-1884] updated ReferenceRegion in python #1892

Merged
merged 1 commit into from Feb 2, 2018

Conversation

5 participants
@akmorrow13
Contributor

akmorrow13 commented Jan 26, 2018

No description provided.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 26, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2609/

Build result: ABORTED

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains ed66b35 # timeout=10Checking out Revision ed66b35 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ed66b35f1a212be8d41703f080721386a9ec0f51First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result ABORTEDNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@akmorrow13

This comment has been minimized.

Contributor

akmorrow13 commented Jan 28, 2018

Jenkins, retest this please.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 28, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2610/

Build result: ABORTED

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains ed66b35 # timeout=10Checking out Revision ed66b35 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ed66b35f1a212be8d41703f080721386a9ec0f51First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result ABORTEDNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 29, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2613/

Build result: FAILURE

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 65c1794 # timeout=10Checking out Revision 65c1794 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 65c179418ba8f697b5a7965f19a9f5280c3f432dFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@coveralls

This comment has been minimized.

coveralls commented Jan 29, 2018

Coverage Status

Coverage decreased (-0.09%) to 82.616% when pulling 959f7f2 on akmorrow13:fix-python-tests into adff336 on bigdatagenomics:master.

@coveralls

This comment has been minimized.

coveralls commented Jan 29, 2018

Coverage Status

Coverage increased (+0.09%) to 82.801% when pulling 2183fd2 on akmorrow13:fix-python-tests into adff336 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 29, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2614/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 29, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2615/
Test PASSed.

@akmorrow13

This comment has been minimized.

Contributor

akmorrow13 commented Jan 29, 2018

Fixes #1889 and #1884

@@ -725,7 +725,7 @@ trait GenomicRDD[T, U <: GenomicRDD[T, U]] extends Logging {
* @return Returns a new GenomicRDD containing only data that overlaps the
* querys region.
*/
def filterByOverlappingRegions(querys: Iterable[ReferenceRegion]): U = {
def filterByOverlappingRegions(querys: java.lang.Iterable[ReferenceRegion]): U = {

This comment has been minimized.

@akmorrow13

akmorrow13 Jan 29, 2018

Contributor

@heuermh we needed to change this to be accessible to python (py4j takes a python iterator and converts it to java ArrayList > java.lang.Iterable). Is there a better way to do this?

This comment has been minimized.

@heuermh

heuermh Jan 29, 2018

Member

The pattern we've been using so far is to have two methods, one marked as Java-friendly. I'd recommend that here, unless the Scala or Java compiler isn't smart enough to distinguish the two different versions.

This comment has been minimized.

@akmorrow13

akmorrow13 Jan 29, 2018

Contributor

@heuermh can you give me an example of this so I can copy the style you have been using?

This comment has been minimized.

@heuermh

heuermh Jan 29, 2018

Member

GenomicRDD has a Java friendly transmute method, https://github.com/bigdatagenomics/adam/blob/master/adam-core/src/main/scala/org/bdgenomics/adam/rdd/GenomicRDD.scala#L221 and others, just search for "friendly" ;)

@akmorrow13

This comment has been minimized.

Contributor

akmorrow13 commented Jan 29, 2018

@heuermh @fnothaft ready for review. Fixes the big broken python bugs. I need some feedback on the java conversion approach for filterByOverlappingRegions()

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 29, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2616/
Test PASSed.

* @param end end position
* @return Reference Region for these genomic coordinates
*/
def fromGenomicRange(referenceName: String, start: Long, end: Long): ReferenceRegion = {

This comment has been minimized.

@heuermh

heuermh Jan 29, 2018

Member

Is this method necessary? In other words, is it difficult to call ReferenceRegion(referenceName, start, end) from Python?

This comment has been minimized.

@akmorrow13

akmorrow13 Jan 29, 2018

Contributor

you cannot call ReferenceRegion.apply(referenceName, start, end) from python.

This comment has been minimized.

@heuermh
@@ -725,7 +725,7 @@ trait GenomicRDD[T, U <: GenomicRDD[T, U]] extends Logging {
* @return Returns a new GenomicRDD containing only data that overlaps the
* querys region.
*/
def filterByOverlappingRegions(querys: Iterable[ReferenceRegion]): U = {
def filterByOverlappingRegions(querys: java.lang.Iterable[ReferenceRegion]): U = {

This comment has been minimized.

@heuermh

heuermh Jan 29, 2018

Member

The pattern we've been using so far is to have two methods, one marked as Java-friendly. I'd recommend that here, unless the Scala or Java compiler isn't smart enough to distinguish the two different versions.

package \
-DskipTests \
-P python \
test \

This comment has been minimized.

@heuermh

heuermh Jan 29, 2018

Member

Don't we need package for the line ./bin/pyadam < scripts/jenkins-test-pyadam.py below (package includes test)?

This comment has been minimized.

@akmorrow13

akmorrow13 Jan 30, 2018

Contributor

Doesn't seem like it, the tests are passing currently.

This comment has been minimized.

@fnothaft

fnothaft Feb 2, 2018

Member

Yeah, we shouldn't need it.

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 30, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2619/
Test PASSed.

@heuermh heuermh requested a review from fnothaft Jan 30, 2018

@heuermh heuermh added this to the 0.24.0 milestone Jan 30, 2018

@heuermh heuermh changed the title from updated ReferenceRegion in python to [ADAM-1889][ADAM-1884] updated ReferenceRegion in python Jan 30, 2018

@AmplabJenkins

This comment has been minimized.

AmplabJenkins commented Jan 30, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2620/
Test PASSed.

@akmorrow13

This comment has been minimized.

Contributor

akmorrow13 commented Feb 2, 2018

ping @fnothaft for review

@fnothaft fnothaft merged commit d066ab1 into bigdatagenomics:master Feb 2, 2018

1 of 2 checks passed

Codacy/PR Quality Review Not so good... This pull request quality could be better.
Details
default Merged build finished.
Details
@fnothaft

This comment has been minimized.

Member

fnothaft commented Feb 2, 2018

Merged! Thanks @akmorrow13!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment