New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ADAM-510] Refactoring RDD function names #979

Closed
wants to merge 1 commit into
base: master
from

Conversation

Projects
None yet
5 participants
@heuermh
Member

heuermh commented Mar 25, 2016

Fixes #910

Summary of initial changes:

class ADAMContext:

adamBamDictionaryLoad  --> loadBamDictionary
adamBamDictionaryLoad --> loadBamDictionary
adamBamLoadReadGroups --> loadBamReadGroups
loadParquet
adamDictionaryLoad --> loadDictionary
loadBam
loadIndexedBam
loadAvro
loadParquetAlignments
loadInterleavedFastq
loadFastq
loadPairedFastq
loadUnpairedFastq
loadVcf
loadParquetGenotypes
loadParquetVariants
loadFasta
loadInterleavedFastqAsFragments
loadGTF --> loadGtf
loadBED --> loadBed
loadNarrowPeak
loadIntervalList
loadParquetFeatures
loadParquetContigFragments
loadParquetFragments
loadVcfAnnotations
loadParquetVariantAnnotations
loadVariantAnnotations
loadFeatures
loadGenes
loadReferenceFile
loadSequence --> loadSequences
loadGenotypes
loadVariants
loadAlignments
loadFragments
loadAlignmentsFromPaths
findFiles

class ADAMRDDFunctions:

adamParquetSave --> saveAsParquet
adamParquetSave --> saveAsParquet

class ADAMSequenceDictionaryRDDAggregator:

getSequenceRecordsFromElement --> getSequenceRecords
adamGetSequenceDictionary --> getSequenceDictionary

class ADAMSpecificRecordSequenceDictionaryRDDAggregator:

getSequenceRecordsFromElement --> getSequenceRecords

object PairingRDD:

rddToPairingRDD

class NucleotideContigFragmentRDDFunctions:

toReads
saveAsFasta
mergeFragments
adamGetReferenceString --> getReferenceString
getSequenceRecordsFromElement --> getSequenceRecords
flankAdjacentFragments
countKmers

class FeatureRDDFunctions:

asGenes --> toGenes
filterByOverlappingRegion

class FragmentRDDFunctions:

toReads
getSequenceRecordsFromElement --> getSequenceRecords

class AlignmentRecordRDDFunctions:

filterByOverlappingRegion
saveAsParquet
adamAlignedRecordSave --> remove method
adamSave --> save
adamSAMString --> saveAsSamString
adamSAMSave --> saveAsSam
getSequenceRecordsFromElement --> getSequenceRecords
adamConvertToSAM --> convertToSam
adamCountKmers --> countKmers
adamSortReadsByReferencePosition --> sortReadsByReferencePosition
adamMarkDuplicates --> markDuplicates
adamBQSR --> bqsr
adamRealignIndels --> realignIndels
adamFlagStat --> flagStat
adamSingleReadBuckets --> groupReads
adamCharacterizeTags --> characterizeTags
adamCharacterizeTagValues --> characterizeTagValues
adamFilterRecordsWithTag --> filterRecordsWithTag
adamSaveAsPairedFastq --> saveAsPairedFastq
adamSaveAsFastq --> saveAsFastq
adamRePairReads --> reassembleReadPairs
toFragments

class VariantContextRDDFunctions:

getSequenceRecordsFromElement --> getSequenceRecords
joinDatabaseVariantAnnotation
getCallsetSamples
saveAsVcf

class GenotypeRDDFunctions:

toVariantContext
filterByOverlappingRegion

Will take a closer look at scaladoc strings and other parts of the codebase in future commits.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 25, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1120/

Build result: FAILURE

GitHub pull request #979 of commit febc58c automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains 1b4c17c6edaae3d5ee128816570e3bb454dd9c6a # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision 1b4c17c6edaae3d5ee128816570e3bb454dd9c6a (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 1b4c17c6edaae3d5ee128816570e3bb454dd9c6aFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Mar 25, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1120/

Build result: FAILURE

GitHub pull request #979 of commit febc58c automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains 1b4c17c6edaae3d5ee128816570e3bb454dd9c6a # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision 1b4c17c6edaae3d5ee128816570e3bb454dd9c6a (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 1b4c17c6edaae3d5ee128816570e3bb454dd9c6aFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Mar 25, 2016

Member

You are a true patriot, @heuermh.

+1

Member

fnothaft commented Mar 25, 2016

You are a true patriot, @heuermh.

+1

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 25, 2016

Member

I've hit some difference between the maven scala compiler and the IDEA one though, I'm afraid.

[ERROR] ./adam-core/src/.../AlignmentRecordRDDFunctions.scala:199:
  error: recursive method saveAsParquet needs result type
[ERROR]     rdd.saveAsParquet(args)

Is there some way I can force a certain implicit? This call should be to ADAMRDDFunctions.saveAsParquet(args: SaveArgs), not recursive on itself.

Member

heuermh commented Mar 25, 2016

I've hit some difference between the maven scala compiler and the IDEA one though, I'm afraid.

[ERROR] ./adam-core/src/.../AlignmentRecordRDDFunctions.scala:199:
  error: recursive method saveAsParquet needs result type
[ERROR]     rdd.saveAsParquet(args)

Is there some way I can force a certain implicit? This call should be to ADAMRDDFunctions.saveAsParquet(args: SaveArgs), not recursive on itself.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Mar 25, 2016

Member

For lack of a better way around the problem above, I've made AlignmentRecordRDDFunctions.saveAsParquet private and renamed it to saveParquet, similar to AlignmentRecordRDDFunctions.saveAvro.

Member

heuermh commented Mar 25, 2016

For lack of a better way around the problem above, I've made AlignmentRecordRDDFunctions.saveAsParquet private and renamed it to saveParquet, similar to AlignmentRecordRDDFunctions.saveAvro.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 25, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1121/
Test PASSed.

AmplabJenkins commented Mar 25, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1121/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 28, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1123/
Test PASSed.

AmplabJenkins commented Mar 28, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1123/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 28, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1124/

Build result: FAILURE

GitHub pull request #979 of commit 30f979a automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains 3eb17d5 # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision 3eb17d5 (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 3eb17d512560267a9ad144d1fb03272899187a2dFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Mar 28, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1124/

Build result: FAILURE

GitHub pull request #979 of commit 30f979a automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains 3eb17d5 # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision 3eb17d5 (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 3eb17d512560267a9ad144d1fb03272899187a2dFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Mar 29, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1126/

Build result: FAILURE

GitHub pull request #979 of commit 6f55f6a automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains bd6707a # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision bd6707a (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f bd6707a304521cf43450c334cae2144e60260e49First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Mar 29, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1126/

Build result: FAILURE

GitHub pull request #979 of commit 6f55f6a automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains bd6707a # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision bd6707a (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f bd6707a304521cf43450c334cae2144e60260e49First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 6, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1134/
Test PASSed.

AmplabJenkins commented Apr 6, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1134/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 6, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1136/

Build result: FAILURE

GitHub pull request #979 of commit 3a7f598 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains 690bd2f # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision 690bd2f (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 690bd2ffa5769c8237c5c8c112647fb7dbf50998First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Apr 6, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1136/

Build result: FAILURE

GitHub pull request #979 of commit 3a7f598 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains 690bd2f # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision 690bd2f (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 690bd2ffa5769c8237c5c8c112647fb7dbf50998First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 6, 2016

Member

Jenkins, retest this please.

Member

heuermh commented Apr 6, 2016

Jenkins, retest this please.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 6, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1137/
Test PASSed.

AmplabJenkins commented Apr 6, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1137/
Test PASSed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 6, 2016

Member

@fnothaft @jpdna @tdanford @ryan-williams this one is ready to review. I know the new unit tests have weak assertions; I wasn't convinced it'd be worth round-tripping them all.

Member

heuermh commented Apr 6, 2016

@fnothaft @jpdna @tdanford @ryan-williams this one is ready to review. I know the new unit tests have weak assertions; I wasn't convinced it'd be worth round-tripping them all.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 13, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1150/

Build result: FAILURE

[...truncated 24 lines...]Triggering ADAM-prb ? 2.6.0,2.10,1.3.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.4.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.4.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.4.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.5.2,centosADAM-prb ? 2.6.0,2.11,1.6.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.5.2,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.3.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.4.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.5.2,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Apr 13, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1150/

Build result: FAILURE

[...truncated 24 lines...]Triggering ADAM-prb ? 2.6.0,2.10,1.3.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.0,centosTriggering ADAM-prb ? 2.6.0,2.10,1.4.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.4.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.4.1,centosTriggering ADAM-prb ? 2.3.0,2.11,1.3.1,centosTriggering ADAM-prb ? 2.3.0,2.10,1.5.2,centosADAM-prb ? 2.6.0,2.11,1.6.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.11,1.5.2,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.6.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.3.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.6.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.10,1.4.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.11,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.4.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,1.3.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.5.2,centos completed with result FAILURENotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 13, 2016

Member

Jenkins, retest this please.

Member

heuermh commented Apr 13, 2016

Jenkins, retest this please.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 13, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1151/
Test PASSed.

AmplabJenkins commented Apr 13, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1151/
Test PASSed.

Show outdated Hide outdated ...ain/scala/org/bdgenomics/adam/rdd/read/AlignmentRecordRDDFunctions.scala
}
import java.io.{ InputStream, OutputStream, StringWriter, Writer }
import java.lang.reflect.InvocationTargetException

This comment has been minimized.

@fnothaft

fnothaft Apr 13, 2016

Member

Ixnay on the spaces.

@fnothaft

fnothaft Apr 13, 2016

Member

Ixnay on the spaces.

Show outdated Hide outdated ...ain/scala/org/bdgenomics/adam/rdd/read/AlignmentRecordRDDFunctions.scala
import org.bdgenomics.adam.rich.RichAlignmentRecord
import org.bdgenomics.adam.util.MapTools
import org.bdgenomics.formats.avro._
import org.seqdoop.hadoop_bam.SAMRecordWritable

This comment has been minimized.

@fnothaft

fnothaft Apr 13, 2016

Member

Drop space.

@fnothaft

fnothaft Apr 13, 2016

Member

Drop space.

Show outdated Hide outdated ...ain/scala/org/bdgenomics/adam/rdd/read/AlignmentRecordRDDFunctions.scala
@@ -672,7 +649,7 @@ class AlignmentRecordRDDFunctions(rdd: RDD[AlignmentRecord]) extends Serializabl
* observations to.
* @return Returns an RDD of recalibrated reads.
*/
def adamBQSR(
def bqsr(

This comment has been minimized.

@fnothaft

fnothaft Apr 13, 2016

Member

If we're renaming this, I might prefer recalibateBaseQualities over bqsr. Thoughts?

@fnothaft

fnothaft Apr 13, 2016

Member

If we're renaming this, I might prefer recalibateBaseQualities over bqsr. Thoughts?

This comment has been minimized.

@heuermh

heuermh Apr 14, 2016

Member

+1

@heuermh
Show outdated Hide outdated ...ain/scala/org/bdgenomics/adam/rdd/read/AlignmentRecordRDDFunctions.scala
@@ -712,7 +689,7 @@ class AlignmentRecordRDDFunctions(rdd: RDD[AlignmentRecord]) extends Serializabl
*
* @return SingleReadBuckets with primary, secondary and unmapped reads
*/
def adamSingleReadBuckets(): RDD[SingleReadBucket] = {
def groupReads(): RDD[SingleReadBucket] = {

This comment has been minimized.

@fnothaft

fnothaft Apr 13, 2016

Member

toSingleReadBuckets?

@fnothaft

fnothaft Apr 13, 2016

Member

toSingleReadBuckets?

This comment has been minimized.

@heuermh

heuermh Apr 14, 2016

Member

I went with the doc comment, Groups all reads by record group and read name. I don't think the word Bucket belongs in software.

@heuermh

heuermh Apr 14, 2016

Member

I went with the doc comment, Groups all reads by record group and read name. I don't think the word Bucket belongs in software.

This comment has been minimized.

@fnothaft

fnothaft Apr 18, 2016

Member

I went with the doc comment, Groups all reads by record group and read name.

Fair, but groupReads is a bit underspecified. I.e., is it grouping by:

  • Read group
  • Position
  • Library
  • Etc.

Since it is grouping reads together by the original sequencing fragment, perhaps groupReadsByFragment?

I don't think the word Bucket belongs in software.

Hah! Why's that?
In any case, perhaps do you want to s/SingleReadBucket/SingleRead/g or something like that?

@fnothaft

fnothaft Apr 18, 2016

Member

I went with the doc comment, Groups all reads by record group and read name.

Fair, but groupReads is a bit underspecified. I.e., is it grouping by:

  • Read group
  • Position
  • Library
  • Etc.

Since it is grouping reads together by the original sequencing fragment, perhaps groupReadsByFragment?

I don't think the word Bucket belongs in software.

Hah! Why's that?
In any case, perhaps do you want to s/SingleReadBucket/SingleRead/g or something like that?

This comment has been minimized.

@heuermh

heuermh Apr 18, 2016

Member

Since it is grouping reads together by the original sequencing fragment, perhaps groupReadsByFragment?

+1

In any case, perhaps do you want to s/SingleReadBucket/SingleRead/g or something like that?

I think I'll wait, I may want to propose other schema/class name changes in a separate pull request.

@heuermh

heuermh Apr 18, 2016

Member

Since it is grouping reads together by the original sequencing fragment, perhaps groupReadsByFragment?

+1

In any case, perhaps do you want to s/SingleReadBucket/SingleRead/g or something like that?

I think I'll wait, I may want to propose other schema/class name changes in a separate pull request.

This comment has been minimized.

@fnothaft

fnothaft Apr 18, 2016

Member

I think I'll wait, I may want to propose other schema/class name changes in a separate pull request.

SGTM!

@fnothaft

fnothaft Apr 18, 2016

Member

I think I'll wait, I may want to propose other schema/class name changes in a separate pull request.

SGTM!

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 13, 2016

Member

Other than a few nits, LGTM!

Member

fnothaft commented Apr 13, 2016

Other than a few nits, LGTM!

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 18, 2016

Member

Addressed review comments and squashed commits.

Could I get a few more +1s? This is a non-binary compatible change that will probably break 99% of external code and scripts.

Member

heuermh commented Apr 18, 2016

Addressed review comments and squashed commits.

Could I get a few more +1s? This is a non-binary compatible change that will probably break 99% of external code and scripts.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 18, 2016

Member

Could I get a few more +1s? This is a non-binary compatible change that will probably break 99% of external code and scripts.

Agreed. I am +1, but it would be good to get more eyes.

Member

fnothaft commented Apr 18, 2016

Could I get a few more +1s? This is a non-binary compatible change that will probably break 99% of external code and scripts.

Agreed. I am +1, but it would be good to get more eyes.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 18, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1154/

Build result: FAILURE

GitHub pull request #979 of commit a649f55 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains 031d36d024933dd66f817dfdae0bca8c431b1be2 # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision 031d36d024933dd66f817dfdae0bca8c431b1be2 (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 031d36d024933dd66f817dfdae0bca8c431b1be2 > git rev-list 86417b1695dc72ab989275a1c69d26f54411bb75 # timeout=10Triggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Apr 18, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1154/

Build result: FAILURE

GitHub pull request #979 of commit a649f55 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains 031d36d024933dd66f817dfdae0bca8c431b1be2 # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision 031d36d024933dd66f817dfdae0bca8c431b1be2 (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f 031d36d024933dd66f817dfdae0bca8c431b1be2 > git rev-list 86417b1695dc72ab989275a1c69d26f54411bb75 # timeout=10Triggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 18, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1155/

Build result: FAILURE

GitHub pull request #979 of commit c217432 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains e201a22 # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision e201a22 (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f e201a22 > git rev-list 031d36d024933dd66f817dfdae0bca8c431b1be2 # timeout=10Triggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

AmplabJenkins commented Apr 18, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1155/

Build result: FAILURE

GitHub pull request #979 of commit c217432 automatically merged.Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'[EnvInject] - Loading node environment variables.Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prb > git rev-parse --is-inside-work-tree # timeout=10Fetching changes from the remote Git repository > git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > git --version # timeout=10 > git fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ > git rev-parse origin/pr/979/merge^{commit} # timeout=10 > git branch -a --contains e201a22 # timeout=10 > git rev-parse remotes/origin/pr/979/merge^{commit} # timeout=10Checking out Revision e201a22 (origin/pr/979/merge) > git config core.sparsecheckout # timeout=10 > git checkout -f e201a22 > git rev-list 031d36d024933dd66f817dfdae0bca8c431b1be2 # timeout=10Triggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Apr 18, 2016

Member

Jenkins, retest this please.

Member

heuermh commented Apr 18, 2016

Jenkins, retest this please.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Apr 18, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1156/
Test PASSed.

AmplabJenkins commented Apr 18, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1156/
Test PASSed.

@tdanford

This comment has been minimized.

Show comment
Hide comment
@tdanford

tdanford Apr 20, 2016

Contributor

+1

Contributor

tdanford commented Apr 20, 2016

+1

@heuermh heuermh changed the title from Refactoring RDD function names to [ADAM-510] Refactoring RDD function names Apr 21, 2016

@ryan-williams

This comment has been minimized.

Show comment
Hide comment
@ryan-williams
Member

ryan-williams commented Apr 22, 2016

+1 thanks @heuermh

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Apr 22, 2016

Member

Thanks to @heuermh for the PR, and to all for the review! I've merged as 70eaa70.

Member

fnothaft commented Apr 22, 2016

Thanks to @heuermh for the PR, and to all for the review! I've merged as 70eaa70.

@fnothaft fnothaft closed this Apr 22, 2016

@heuermh heuermh deleted the heuermh:functions branch Jun 28, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment