[ADAM-1889][ADAM-1884] updated ReferenceRegion in python #1892

akmorrow13 · Jan 26, 2018

No description provided.

AmplabJenkins · Jan 26, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2609/

Build result: ABORTED

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains ed66b35 # timeout=10Checking out Revision ed66b35 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ed66b35f1a212be8d41703f080721386a9ec0f51First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result ABORTEDNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

akmorrow13 · Jan 28, 2018

Jenkins, retest this please.

AmplabJenkins · Jan 28, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2610/

Build result: ABORTED

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains ed66b35 # timeout=10Checking out Revision ed66b35 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ed66b35f1a212be8d41703f080721386a9ec0f51First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result ABORTEDNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · Jan 29, 2018

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2613/

Build result: FAILURE

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 65c1794 # timeout=10Checking out Revision 65c1794 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 65c179418ba8f697b5a7965f19a9f5280c3f432dFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

coveralls · Jan 29, 2018

Coverage decreased (-0.09%) to 82.616% when pulling 959f7f2 on akmorrow13:fix-python-tests into adff336 on bigdatagenomics:master.

coveralls · Jan 29, 2018

Coverage increased (+0.09%) to 82.801% when pulling 2183fd2 on akmorrow13:fix-python-tests into adff336 on bigdatagenomics:master.

AmplabJenkins · Jan 29, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2614/
Test PASSed.

AmplabJenkins · Jan 29, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2615/
Test PASSed.

akmorrow13 · Jan 29, 2018

Fixes #1889 and #1884

akmorrow13 · Jan 29, 2018

akmorrow13 reviewed Jan 29, 2018

View changes

akmorrow13 · Jan 29, 2018

adam-core/src/main/scala/org/bdgenomics/adam/rdd/GenomicRDD.scala

@@ -725,7 +725,7 @@ trait GenomicRDD[T, U <: GenomicRDD[T, U]] extends Logging {
   * @return Returns a new GenomicRDD containing only data that overlaps the
   *   querys region.
   */
-  def filterByOverlappingRegions(querys: Iterable[ReferenceRegion]): U = {
+  def filterByOverlappingRegions(querys: java.lang.Iterable[ReferenceRegion]): U = {


@heuermh we needed to change this to be accessible to python (py4j takes a python iterator and converts it to java ArrayList > java.lang.Iterable). Is there a better way to do this?

The pattern we've been using so far is to have two methods, one marked as Java-friendly. I'd recommend that here, unless the Scala or Java compiler isn't smart enough to distinguish the two different versions.

@heuermh can you give me an example of this so I can copy the style you have been using?

GenomicRDD has a Java friendly transmute method, https://github.com/bigdatagenomics/adam/blob/master/adam-core/src/main/scala/org/bdgenomics/adam/rdd/GenomicRDD.scala#L221 and others, just search for "friendly" ;)

akmorrow13 · Jan 29, 2018

@heuermh @fnothaft ready for review. Fixes the big broken python bugs. I need some feedback on the java conversion approach for filterByOverlappingRegions()

AmplabJenkins · Jan 29, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2616/
Test PASSed.

heuermh · Jan 29, 2018

heuermh requested changes Jan 29, 2018

View changes

heuermh · Jan 29, 2018

adam-core/src/main/scala/org/bdgenomics/adam/models/ReferenceRegion.scala

+   * @param end end position
+   * @return Reference Region for these genomic coordinates
+   */
+  def fromGenomicRange(referenceName: String, start: Long, end: Long): ReferenceRegion = {


Is this method necessary? In other words, is it difficult to call ReferenceRegion(referenceName, start, end) from Python?

you cannot call ReferenceRegion.apply(referenceName, start, end) from python.

heuermh · Jan 29, 2018

adam-core/src/main/scala/org/bdgenomics/adam/rdd/GenomicRDD.scala

@@ -725,7 +725,7 @@ trait GenomicRDD[T, U <: GenomicRDD[T, U]] extends Logging {
   * @return Returns a new GenomicRDD containing only data that overlaps the
   *   querys region.
   */
-  def filterByOverlappingRegions(querys: Iterable[ReferenceRegion]): U = {
+  def filterByOverlappingRegions(querys: java.lang.Iterable[ReferenceRegion]): U = {


The pattern we've been using so far is to have two methods, one marked as Java-friendly. I'd recommend that here, unless the Scala or Java compiler isn't smart enough to distinguish the two different versions.

heuermh · Jan 29, 2018

scripts/jenkins-test

-		package \
-		-DskipTests \
+		-P python \
+		test \


Don't we need package for the line ./bin/pyadam < scripts/jenkins-test-pyadam.py below (package includes test)?

Doesn't seem like it, the tests are passing currently.

Yeah, we shouldn't need it.

AmplabJenkins · Jan 30, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2619/
Test PASSed.

heuermh · Jan 30, 2018

heuermh approved these changes Jan 30, 2018

View changes

AmplabJenkins · Jan 30, 2018

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2620/
Test PASSed.

akmorrow13 · Feb 2, 2018

ping @fnothaft for review

fnothaft · Feb 2, 2018

fnothaft approved these changes Feb 2, 2018

View changes

fnothaft · Feb 2, 2018

Merged! Thanks @akmorrow13!

heuermh requested a review from fnothaft Jan 30, 2018

heuermh added this to the 0.24.0 milestone Jan 30, 2018

heuermh changed the title from updated ReferenceRegion in python to [ADAM-1889][ADAM-1884] updated ReferenceRegion in python Jan 30, 2018

updated ReferenceRegion in python

Loading status checks…

959f7f2

akmorrow13 referenced this pull request Feb 2, 2018
Closed
[ADAM-1889] Run R/Python tests. #1890

fnothaft merged commit d066ab1 into bigdatagenomics:master Feb 2, 2018
1 of 2 checks passed

1 of 2 checks passed

Codacy/PR Quality Review Not so good... This pull request quality could be better.
Details

default Merged build finished.
Details

This was referenced Feb 2, 2018

Closed

ReferenceRegion in python does not exist #1884

Closed

Jenkins does not run python or R tests #1889

heuermh added this to Completed in Release 0.24.0 Feb 10, 2018

bigdatagenomics/adam

Join GitHub today

[ADAM-1889][ADAM-1884] updated ReferenceRegion in python #1892

Conversation

akmorrow13 commented Jan 26, 2018

This comment has been minimized.

AmplabJenkins commented Jan 26, 2018

Build result: ABORTED

This comment has been minimized.

akmorrow13 commented Jan 28, 2018

This comment has been minimized.

AmplabJenkins commented Jan 28, 2018

Build result: ABORTED

This comment has been minimized.

AmplabJenkins commented Jan 29, 2018

Build result: FAILURE

This comment has been minimized.

coveralls commented Jan 29, 2018 • edited

This comment has been minimized.

coveralls commented Jan 29, 2018

This comment has been minimized.

AmplabJenkins commented Jan 29, 2018

This comment has been minimized.

AmplabJenkins commented Jan 29, 2018

This comment has been minimized.

akmorrow13 commented Jan 29, 2018

akmorrow13 reviewed Jan 29, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

akmorrow13 commented Jan 29, 2018

This comment has been minimized.

AmplabJenkins commented Jan 29, 2018

heuermh requested changes Jan 29, 2018 View changes

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

AmplabJenkins commented Jan 30, 2018

heuermh approved these changes Jan 30, 2018 View changes

heuermh requested a review from fnothaft Jan 30, 2018

heuermh added this to the 0.24.0 milestone Jan 30, 2018

heuermh changed the title from updated ReferenceRegion in python to [ADAM-1889][ADAM-1884] updated ReferenceRegion in python Jan 30, 2018

This comment has been minimized.

AmplabJenkins commented Jan 30, 2018

This comment has been minimized.

akmorrow13 commented Feb 2, 2018

akmorrow13 referenced this pull request Feb 2, 2018

fnothaft approved these changes Feb 2, 2018 View changes

Hide details View details fnothaft merged commit d066ab1 into bigdatagenomics:master Feb 2, 2018 1 of 2 checks passed

1 of 2 checks passed

This comment has been minimized.

fnothaft commented Feb 2, 2018

This was referenced Feb 2, 2018

heuermh added this to Completed in Release 0.24.0 Feb 10, 2018

coveralls commented Jan 29, 2018 •

edited

akmorrow13 reviewed Jan 29, 2018

View changes

heuermh requested changes Jan 29, 2018

View changes

heuermh approved these changes Jan 30, 2018

View changes

fnothaft approved these changes Feb 2, 2018

View changes

fnothaft merged commit `d066ab1` into bigdatagenomics:master Feb 2, 2018
1 of 2 checks passed