Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add quality score binner #1485

Merged

Conversation

fnothaft
Copy link
Member

@fnothaft fnothaft commented Apr 11, 2017

Resolves #1462. WIP. Still need to:

  • Add docs
  • Characterize
  • Add to fragment path

@coveralls
Copy link

coveralls commented Apr 11, 2017

Coverage Status

Coverage increased (+0.1%) to 81.785% when pulling 2d9cb2c on fnothaft:issues/1462-quality-score-binner into 93b32c6 on bigdatagenomics:master.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1941/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1485/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains d0fb3edd7021762f878ef7a5ab66ab37dd25c38e # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1485/merge^{commit} # timeout=10Checking out Revision d0fb3edd7021762f878ef7a5ab66ab37dd25c38e (origin/pr/1485/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f d0fb3edd7021762f878ef7a5ab66ab37dd25c38eFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft
Copy link
Member Author

Jenkins, retest this please.

@coveralls
Copy link

coveralls commented Apr 11, 2017

Coverage Status

Coverage increased (+0.1%) to 81.785% when pulling 2d9cb2c on fnothaft:issues/1462-quality-score-binner into 93b32c6 on bigdatagenomics:master.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1942/
Test PASSed.

Copy link
Member

@heuermh heuermh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! Nice test coverage. Does it help performance much?

@@ -115,6 +115,8 @@ class TransformArgs extends Args4jBase with ADAMSaveAnyArgs with ParquetArgs {
var mdTagsFragmentSize: Long = 1000000L
@Args4jOption(required = false, name = "-md_tag_overwrite", usage = "When adding MD tags to reads, overwrite existing incorrect tags.")
var mdTagsOverwrite: Boolean = false
@Args4jOption(required = false, name = "-bin_quality_scores", usage = "Rewrites quality scores of reads into bins. User must provide bin description string.")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about
Rewrites quality scores of reads into bins from a string of bin descriptions, e.g. 0,20,10;20,40,30.

// if we have multiple bins, validate them
// - check that we don't have gaps between bins
// - check that we don't have overlapping bins
if (bins.size > 1) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

would this check be better in QualityScoreBin.apply?

@heuermh
Copy link
Member

heuermh commented May 3, 2017

I'd be ok merging this with the doc string fix suggested above

@fnothaft fnothaft added this to the 0.23.0 milestone May 11, 2017
Resolves bigdatagenomics#1462. Adds an API for binning quality scores to `AlignmentRecordRDD`
and `FragmentRDD`. Exposes this on the `Transform` CLI.
@fnothaft fnothaft force-pushed the issues/1462-quality-score-binner branch from 2d9cb2c to a089670 Compare May 12, 2017 07:11
@fnothaft
Copy link
Member Author

@heuermh addressed the doc issue. This is good to go from my side.

@coveralls
Copy link

coveralls commented May 12, 2017

Coverage Status

Coverage increased (+0.1%) to 82.015% when pulling a089670 on fnothaft:issues/1462-quality-score-binner into 0dd09b2 on bigdatagenomics:master.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1989/
Test PASSed.

@heuermh heuermh merged commit 18191f9 into bigdatagenomics:master May 12, 2017
@heuermh
Copy link
Member

heuermh commented May 12, 2017

Thank you, @fnothaft

@fnothaft fnothaft deleted the issues/1462-quality-score-binner branch May 21, 2017 22:57
fnothaft added a commit to fnothaft/adam that referenced this pull request May 21, 2017
…formFragments`

Resolves bigdatagenomics#1359. Also, propegates the quality score binner (bigdatagenomics#1485) up to the
`transformFragments` CLI.
fnothaft added a commit to fnothaft/adam that referenced this pull request May 26, 2017
…formFragments`

Resolves bigdatagenomics#1359. Also, propegates the quality score binner (bigdatagenomics#1485) up to the
`transformFragments` CLI.
heuermh pushed a commit that referenced this pull request May 30, 2017
…formFragments`

Resolves #1359. Also, propegates the quality score binner (#1485) up to the
`transformFragments` CLI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants