Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add quality score binner #1485

Merged

Conversation

@fnothaft
Copy link
Member

@fnothaft fnothaft commented Apr 11, 2017

Resolves #1462. WIP. Still need to:

  • Add docs
  • Characterize
  • Add to fragment path
@coveralls
Copy link

@coveralls coveralls commented Apr 11, 2017

Coverage Status

Coverage increased (+0.1%) to 81.785% when pulling 2d9cb2c on fnothaft:issues/1462-quality-score-binner into 93b32c6 on bigdatagenomics:master.

@AmplabJenkins
Copy link

@AmplabJenkins AmplabJenkins commented Apr 11, 2017

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1941/

Build result: FAILURE

[...truncated 16 lines...] > /home/jenkins/git2/bin/git rev-parse origin/pr/1485/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains d0fb3edd7021762f878ef7a5ab66ab37dd25c38e # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1485/merge^{commit} # timeout=10Checking out Revision d0fb3edd7021762f878ef7a5ab66ab37dd25c38e (origin/pr/1485/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f d0fb3edd7021762f878ef7a5ab66ab37dd25c38eFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.3.0,2.11,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.3.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.11,2.0.0,centosTriggering ADAM-prb ? 2.6.0,2.10,2.0.0,centosTriggering ADAM-prb ? 2.3.0,2.10,1.6.1,centosTriggering ADAM-prb ? 2.6.0,2.11,1.6.1,centosADAM-prb ? 2.3.0,2.11,1.6.1,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.3.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.3.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,2.0.0,centos completed with result SUCCESSADAM-prb ? 2.6.0,2.10,2.0.0,centos completed with result FAILUREADAM-prb ? 2.3.0,2.10,1.6.1,centos completed with result FAILUREADAM-prb ? 2.6.0,2.11,1.6.1,centos completed with result SUCCESSNotifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@fnothaft
Copy link
Member Author

@fnothaft fnothaft commented Apr 11, 2017

Jenkins, retest this please.

@coveralls
Copy link

@coveralls coveralls commented Apr 11, 2017

Coverage Status

Coverage increased (+0.1%) to 81.785% when pulling 2d9cb2c on fnothaft:issues/1462-quality-score-binner into 93b32c6 on bigdatagenomics:master.

@AmplabJenkins
Copy link

@AmplabJenkins AmplabJenkins commented Apr 11, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1942/
Test PASSed.

Copy link
Member

@heuermh heuermh left a comment

Looks good! Nice test coverage. Does it help performance much?

@@ -115,6 +115,8 @@ class TransformArgs extends Args4jBase with ADAMSaveAnyArgs with ParquetArgs {
var mdTagsFragmentSize: Long = 1000000L
@Args4jOption(required = false, name = "-md_tag_overwrite", usage = "When adding MD tags to reads, overwrite existing incorrect tags.")
var mdTagsOverwrite: Boolean = false
@Args4jOption(required = false, name = "-bin_quality_scores", usage = "Rewrites quality scores of reads into bins. User must provide bin description string.")

This comment has been minimized.

@heuermh

heuermh Apr 12, 2017
Member

How about
Rewrites quality scores of reads into bins from a string of bin descriptions, e.g. 0,20,10;20,40,30.

// if we have multiple bins, validate them
// - check that we don't have gaps between bins
// - check that we don't have overlapping bins
if (bins.size > 1) {

This comment has been minimized.

@heuermh

heuermh Apr 12, 2017
Member

would this check be better in QualityScoreBin.apply?

@heuermh
Copy link
Member

@heuermh heuermh commented May 3, 2017

I'd be ok merging this with the doc string fix suggested above

@fnothaft fnothaft added this to the 0.23.0 milestone May 11, 2017
Resolves #1462. Adds an API for binning quality scores to `AlignmentRecordRDD`
and `FragmentRDD`. Exposes this on the `Transform` CLI.
@fnothaft fnothaft force-pushed the fnothaft:issues/1462-quality-score-binner branch from 2d9cb2c to a089670 May 12, 2017
@fnothaft
Copy link
Member Author

@fnothaft fnothaft commented May 12, 2017

@heuermh addressed the doc issue. This is good to go from my side.

@coveralls
Copy link

@coveralls coveralls commented May 12, 2017

Coverage Status

Coverage increased (+0.1%) to 82.015% when pulling a089670 on fnothaft:issues/1462-quality-score-binner into 0dd09b2 on bigdatagenomics:master.

@AmplabJenkins
Copy link

@AmplabJenkins AmplabJenkins commented May 12, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1989/
Test PASSed.

@heuermh heuermh merged commit 18191f9 into bigdatagenomics:master May 12, 2017
2 of 3 checks passed
2 of 3 checks passed
codacy/pr Not so good... This pull request quality could be better.
Details
coverage/coveralls Coverage increased (+0.1%) to 82.015%
Details
default Merged build finished.
Details
@heuermh
Copy link
Member

@heuermh heuermh commented May 12, 2017

Thank you, @fnothaft

@fnothaft fnothaft deleted the fnothaft:issues/1462-quality-score-binner branch May 21, 2017
fnothaft added a commit to fnothaft/adam that referenced this pull request May 21, 2017
…formFragments`

Resolves bigdatagenomics#1359. Also, propegates the quality score binner (bigdatagenomics#1485) up to the
`transformFragments` CLI.
fnothaft added a commit to fnothaft/adam that referenced this pull request May 26, 2017
…formFragments`

Resolves bigdatagenomics#1359. Also, propegates the quality score binner (bigdatagenomics#1485) up to the
`transformFragments` CLI.
heuermh added a commit that referenced this pull request May 30, 2017
…formFragments`

Resolves #1359. Also, propegates the quality score binner (#1485) up to the
`transformFragments` CLI.
@heuermh heuermh added this to Completed in Release 0.23.0 May 30, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Linked issues

Successfully merging this pull request may close these issues.

None yet

4 participants
You can’t perform that action at this time.