[ADAM-1889][ADAM-1884] updated ReferenceRegion in python #1892

akmorrow13 · 2018-01-26T18:00:16Z

No description provided.

AmplabJenkins · 2018-01-26T18:48:16Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2609/

Build result: ABORTED

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains ed66b35 # timeout=10Checking out Revision ed66b35 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ed66b35f1a212be8d41703f080721386a9ec0f51First time build. Skipping changelog.Triggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result ABORTEDNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

akmorrow13 · 2018-01-28T18:15:40Z

Jenkins, retest this please.

AmplabJenkins · 2018-01-28T19:03:17Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2610/

Build result: ABORTED

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains ed66b35 # timeout=10Checking out Revision ed66b35 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f ed66b35f1a212be8d41703f080721386a9ec0f51First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result ABORTEDADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result ABORTEDNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins · 2018-01-29T00:03:56Z

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2613/

Build result: FAILURE

[...truncated 7 lines...] > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1892/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a -v --no-abbrev --contains 65c1794 # timeout=10Checking out Revision 65c1794 (origin/pr/1892/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 65c179418ba8f697b5a7965f19a9f5280c3f432dFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.2,2.10,2.2.1,centosTriggering ADAM-prb ? 2.6.2,2.11,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.10,2.2.1,centosTriggering ADAM-prb ? 2.7.3,2.11,2.2.1,centosADAM-prb ? 2.6.2,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.6.2,2.11,2.2.1,centos completed with result SUCCESSADAM-prb ? 2.7.3,2.10,2.2.1,centos completed with result FAILUREADAM-prb ? 2.7.3,2.11,2.2.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

coveralls · 2018-01-29T00:20:50Z

Coverage decreased (-0.09%) to 82.616% when pulling 959f7f2 on akmorrow13:fix-python-tests into adff336 on bigdatagenomics:master.

coveralls · 2018-01-29T00:20:50Z

Coverage increased (+0.09%) to 82.801% when pulling 2183fd2 on akmorrow13:fix-python-tests into adff336 on bigdatagenomics:master.

AmplabJenkins · 2018-01-29T00:42:07Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2614/
Test PASSed.

AmplabJenkins · 2018-01-29T01:14:16Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2615/
Test PASSed.

akmorrow13 · 2018-01-29T01:15:28Z

Fixes #1889 and #1884

akmorrow13 · 2018-01-29T01:17:04Z

adam-core/src/main/scala/org/bdgenomics/adam/rdd/GenomicRDD.scala

@@ -725,7 +725,7 @@ trait GenomicRDD[T, U <: GenomicRDD[T, U]] extends Logging {
   * @return Returns a new GenomicRDD containing only data that overlaps the
   *   querys region.
   */
-  def filterByOverlappingRegions(querys: Iterable[ReferenceRegion]): U = {
+  def filterByOverlappingRegions(querys: java.lang.Iterable[ReferenceRegion]): U = {


@heuermh we needed to change this to be accessible to python (py4j takes a python iterator and converts it to java ArrayList > java.lang.Iterable). Is there a better way to do this?

The pattern we've been using so far is to have two methods, one marked as Java-friendly. I'd recommend that here, unless the Scala or Java compiler isn't smart enough to distinguish the two different versions.

@heuermh can you give me an example of this so I can copy the style you have been using?

GenomicRDD has a Java friendly transmute method, https://github.com/bigdatagenomics/adam/blob/master/adam-core/src/main/scala/org/bdgenomics/adam/rdd/GenomicRDD.scala#L221 and others, just search for "friendly" ;)

akmorrow13 · 2018-01-29T01:17:54Z

@heuermh @fnothaft ready for review. Fixes the big broken python bugs. I need some feedback on the java conversion approach for filterByOverlappingRegions()

AmplabJenkins · 2018-01-29T01:50:28Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2616/
Test PASSed.

heuermh · 2018-01-29T17:11:47Z

adam-core/src/main/scala/org/bdgenomics/adam/models/ReferenceRegion.scala

+   * @param end end position
+   * @return Reference Region for these genomic coordinates
+   */
+  def fromGenomicRange(referenceName: String, start: Long, end: Long): ReferenceRegion = {


Is this method necessary? In other words, is it difficult to call ReferenceRegion(referenceName, start, end) from Python?

you cannot call ReferenceRegion.apply(referenceName, start, end) from python.

heuermh · 2018-01-29T17:11:47Z

adam-core/src/main/scala/org/bdgenomics/adam/rdd/GenomicRDD.scala

@@ -725,7 +725,7 @@ trait GenomicRDD[T, U <: GenomicRDD[T, U]] extends Logging {
   * @return Returns a new GenomicRDD containing only data that overlaps the
   *   querys region.
   */
-  def filterByOverlappingRegions(querys: Iterable[ReferenceRegion]): U = {
+  def filterByOverlappingRegions(querys: java.lang.Iterable[ReferenceRegion]): U = {


The pattern we've been using so far is to have two methods, one marked as Java-friendly. I'd recommend that here, unless the Scala or Java compiler isn't smart enough to distinguish the two different versions.

heuermh · 2018-01-29T17:11:47Z

scripts/jenkins-test

-		package \
-		-DskipTests \
+		-P python \
+		test \


Don't we need package for the line ./bin/pyadam < scripts/jenkins-test-pyadam.py below (package includes test)?

Doesn't seem like it, the tests are passing currently.

Yeah, we shouldn't need it.

AmplabJenkins · 2018-01-30T01:20:53Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2619/
Test PASSed.

AmplabJenkins · 2018-01-30T04:27:15Z

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2620/
Test PASSed.

akmorrow13 · 2018-02-02T17:21:43Z

ping @fnothaft for review

fnothaft · 2018-02-02T19:40:55Z

Merged! Thanks @akmorrow13!

akmorrow13 force-pushed the akmorrow13:fix-python-tests branch from 2183fd2 to 95e7109 Jan 29, 2018

akmorrow13 reviewed Jan 29, 2018

View changes

heuermh requested changes Jan 29, 2018

View changes

heuermh approved these changes Jan 30, 2018

View changes

heuermh requested a review from fnothaft Jan 30, 2018

heuermh added this to the 0.24.0 milestone Jan 30, 2018

heuermh changed the title ~~updated ReferenceRegion in python~~ [ADAM-1889][ADAM-1884] updated ReferenceRegion in python Jan 30, 2018

updated ReferenceRegion in python

Loading status checks…

959f7f2

akmorrow13 force-pushed the akmorrow13:fix-python-tests branch from e3540ac to 959f7f2 Jan 30, 2018

akmorrow13 mentioned this pull request Feb 2, 2018

[ADAM-1889] Run R/Python tests. #1890

Closed

fnothaft approved these changes Feb 2, 2018

View changes

fnothaft merged commit d066ab1 into bigdatagenomics:master Feb 2, 2018
1 of 2 checks passed

1 of 2 checks passed

Codacy/PR Quality Review Not so good... This pull request quality could be better.
Details

default Merged build finished.
Details

This was referenced Feb 2, 2018

ReferenceRegion in python does not exist #1884

Closed

Jenkins does not run python or R tests #1889

Closed

heuermh added this to Completed in Release 0.24.0 Feb 10, 2018

bigdatagenomics / adam

[ADAM-1889][ADAM-1884] updated ReferenceRegion in python #1892

[ADAM-1889][ADAM-1884] updated ReferenceRegion in python #1892

akmorrow13 commented Jan 26, 2018

AmplabJenkins commented Jan 26, 2018

akmorrow13 commented Jan 28, 2018

AmplabJenkins commented Jan 28, 2018

AmplabJenkins commented Jan 29, 2018

coveralls commented Jan 29, 2018 •

edited

coveralls commented Jan 29, 2018

AmplabJenkins commented Jan 29, 2018

AmplabJenkins commented Jan 29, 2018

akmorrow13 commented Jan 29, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

akmorrow13 commented Jan 29, 2018

AmplabJenkins commented Jan 29, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

AmplabJenkins commented Jan 30, 2018

AmplabJenkins commented Jan 30, 2018

akmorrow13 commented Feb 2, 2018

fnothaft commented Feb 2, 2018

bigdatagenomics / adam

[ADAM-1889][ADAM-1884] updated ReferenceRegion in python #1892

[ADAM-1889][ADAM-1884] updated ReferenceRegion in python #1892

Conversation

akmorrow13 commented Jan 26, 2018

AmplabJenkins commented Jan 26, 2018

Build result: ABORTED

akmorrow13 commented Jan 28, 2018

AmplabJenkins commented Jan 28, 2018

Build result: ABORTED

AmplabJenkins commented Jan 29, 2018

Build result: FAILURE

coveralls commented Jan 29, 2018 • edited

coveralls commented Jan 29, 2018

AmplabJenkins commented Jan 29, 2018

AmplabJenkins commented Jan 29, 2018

akmorrow13 commented Jan 29, 2018

This comment has been minimized.

akmorrow13 Jan 29, 2018 Author Contributor

This comment has been minimized.

heuermh Jan 29, 2018 Member

This comment has been minimized.

akmorrow13 Jan 29, 2018 Author Contributor

This comment has been minimized.

heuermh Jan 29, 2018 Member

akmorrow13 commented Jan 29, 2018

AmplabJenkins commented Jan 29, 2018

This comment has been minimized.

heuermh Jan 29, 2018 Member

This comment has been minimized.

akmorrow13 Jan 29, 2018 Author Contributor

This comment has been minimized.

heuermh Jan 29, 2018 Member

This comment has been minimized.

heuermh Jan 29, 2018 Member

This comment has been minimized.

heuermh Jan 29, 2018 Member

This comment has been minimized.

akmorrow13 Jan 30, 2018 Author Contributor

This comment has been minimized.

fnothaft Feb 2, 2018 Member

AmplabJenkins commented Jan 30, 2018

AmplabJenkins commented Jan 30, 2018

akmorrow13 commented Feb 2, 2018

fnothaft commented Feb 2, 2018

coveralls commented Jan 29, 2018 •

edited

akmorrow13 Jan 29, 2018
Author Contributor

heuermh Jan 29, 2018
Member

akmorrow13 Jan 29, 2018
Author Contributor

heuermh Jan 29, 2018
Member

heuermh Jan 29, 2018
Member

akmorrow13 Jan 29, 2018
Author Contributor

heuermh Jan 29, 2018
Member

heuermh Jan 29, 2018
Member

heuermh Jan 29, 2018
Member

akmorrow13 Jan 30, 2018
Author Contributor

fnothaft Feb 2, 2018
Member