Clean up CLI operation categories and names, and add documentation for CLI #1274

Merged
merged 1 commit into from Nov 17, 2016

Conversation

Projects
None yet
4 participants
@fnothaft
Member

fnothaft commented Nov 16, 2016

Builds on #1262.

  • Removed CalculateDepth command.
  • Clean up argument doc in TransformFeatures.
  • Move OUTPUT to an option for Flagstat.
  • Remove some output saving arguments in the View command that don't exactly make sense.
  • Remove unused Vcf2ADAM/ADAM2Vcf -dict option and org.bdgenomics.adam.cli.DictionaryCommand trait.
  • Remove WigFix2ADAM command.
  • Move away from ParquetLoadSaveArgs.
  • Move reads2coverage to actions.
  • Reads2Coverage does not only write to Parquet, so clean up docs.
  • Clean up fragments<->reads argument names.
@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 16, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1614/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1274/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains e0e024bb2b997957288d57ae1908e6b9339253b6 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1274/merge^{commit} # timeout=10Checking out Revision e0e024bb2b997957288d57ae1908e6b9339253b6 (origin/pr/1274/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f e0e024bb2b997957288d57ae1908e6b9339253b6First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1614/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1274/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains e0e024bb2b997957288d57ae1908e6b9339253b6 # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1274/merge^{commit} # timeout=10Checking out Revision e0e024bb2b997957288d57ae1908e6b9339253b6 (origin/pr/1274/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f e0e024bb2b997957288d57ae1908e6b9339253b6First time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 16, 2016

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1615/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1274/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 8103cd4cab0ef23be24b0e4d90989f4400e457ba # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1274/merge^{commit} # timeout=10Checking out Revision 8103cd4cab0ef23be24b0e4d90989f4400e457ba (origin/pr/1274/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 8103cd4cab0ef23be24b0e4d90989f4400e457baFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1615/

Build result: FAILURE

[...truncated 3 lines...]Building remotely on amp-jenkins-worker-05 (centos spark-test) in workspace /home/jenkins/workspace/ADAM-prbWiping out workspace first.Cloning the remote Git repositoryCloning repository https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git init /home/jenkins/workspace/ADAM-prb # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git --version # timeout=10 > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/heads/:refs/remotes/origin/ # timeout=15 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10 > /home/jenkins/git2/bin/git config --add remote.origin.fetch +refs/heads/:refs/remotes/origin/ # timeout=10 > /home/jenkins/git2/bin/git config remote.origin.url https://github.com/bigdatagenomics/adam.git # timeout=10Fetching upstream changes from https://github.com/bigdatagenomics/adam.git > /home/jenkins/git2/bin/git -c core.askpass=true fetch --tags --progress https://github.com/bigdatagenomics/adam.git +refs/pull/:refs/remotes/origin/pr/ # timeout=15 > /home/jenkins/git2/bin/git rev-parse origin/pr/1274/merge^{commit} # timeout=10 > /home/jenkins/git2/bin/git branch -a --contains 8103cd4cab0ef23be24b0e4d90989f4400e457ba # timeout=10 > /home/jenkins/git2/bin/git rev-parse remotes/origin/pr/1274/merge^{commit} # timeout=10Checking out Revision 8103cd4cab0ef23be24b0e4d90989f4400e457ba (origin/pr/1274/merge) > /home/jenkins/git2/bin/git config core.sparsecheckout # timeout=10 > /home/jenkins/git2/bin/git checkout -f 8103cd4cab0ef23be24b0e4d90989f4400e457baFirst time build. Skipping changelog.Triggering ADAM-prb ? 2.6.0,2.11,1.5.2,centosTriggering ADAM-prb ? 2.6.0,2.10,1.5.2,centosTouchstone configurations resulted in FAILURE, so aborting...Notifying endpoint 'HTTP:https://webhooks.gitter.im/e/ac8bb6e9f53357bc8aa8'
Test FAILed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 16, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1616/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1616/
Test PASSed.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 17, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1618/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1618/
Test PASSed.

+
+#### Legacy output options {#legacy-output}
+
+Several tools in ADAM support saving back to legacy genomics output formats. Any

This comment has been minimized.

@heuermh

heuermh Nov 17, 2016

Member

legacy genomics output formats, scandalous! :)

@heuermh

heuermh Nov 17, 2016

Member

legacy genomics output formats, scandalous! :)

docs/source/50_cli.md
+required arguments as `vcf2adam`. `anno2adam` takes the [default
+options](#default-args), and one additional option:
+
+* `-current_db`: A path to an existing Parquet file of VariantAnnotations. These

This comment has been minimized.

@heuermh

heuermh Nov 17, 2016

Member

-current_db is a poorly named argument.

@heuermh

heuermh Nov 17, 2016

Member

-current_db is a poorly named argument.

docs/source/50_cli.md
+Dumps a Parquet file to either the console or a text file as
+[JSON](http://www.json.org). Takes one required argument:
+
+1. `FILE(S)`: The file paths to load. This must be a Parquet formatted file.

This comment has been minimized.

@heuermh

heuermh Nov 17, 2016

Member

→ These must be a Parquet formatted files.

@heuermh

heuermh Nov 17, 2016

Member

→ These must be a Parquet formatted files.

Clean up CLI operation categories and names, and add documentation fo…
…r CLI.

* Removed CalculateDepth command.
* Clean up argument doc in TransformFeatures.
* Move OUTPUT to an option for Flagstat.
* Remove some output saving arguments in the View command that don't exactly
  make sense.
* Remove unused Vcf2ADAM/ADAM2Vcf -dict option and
  `org.bdgenomics.adam.cli.DictionaryCommand` trait.
* Remove WigFix2ADAM command.
* Move away from ParquetLoadSaveArgs.
* Move reads2coverage to actions.
* Reads2Coverage does not only write to Parquet, so clean up docs.
* Clean up fragments<->reads argument names.
@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Nov 17, 2016

Member

Addressed review comments.

Member

fnothaft commented Nov 17, 2016

Addressed review comments.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Nov 17, 2016

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1619/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/1619/
Test PASSed.

@heuermh heuermh merged commit 8fef9a7 into bigdatagenomics:master Nov 17, 2016

1 check passed

default Merged build finished.
Details
@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Nov 17, 2016

Member

Thank you, @fnothaft!

Member

heuermh commented Nov 17, 2016

Thank you, @fnothaft!

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Dec 20, 2016

Contributor

Shucks, missed this PR. I use the wigfix2bed CLI command in the Spark book. Out of scope for ADAM?

Contributor

laserson commented Dec 20, 2016

Shucks, missed this PR. I use the wigfix2bed CLI command in the Spark book. Out of scope for ADAM?

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Dec 20, 2016

Contributor

I'll use BEDOPS, which seems to have a pre-written tool for this

Contributor

laserson commented Dec 20, 2016

I'll use BEDOPS, which seems to have a pre-written tool for this

@heuermh

This comment has been minimized.

Show comment
Hide comment
@heuermh

heuermh Dec 21, 2016

Member

@laserson We removed the wigfix2bed because it felt too small for a top-level command. We've also revamped feature support since, so there is no reason to limit export to BED format.

If wigfix is something useful to have, perhaps we should support it as a feature format, and then conversions to/from will be supported by the transformFeatures command. Create an issue if you feel so.

Member

heuermh commented Dec 21, 2016

@laserson We removed the wigfix2bed because it felt too small for a top-level command. We've also revamped feature support since, so there is no reason to limit export to BED format.

If wigfix is something useful to have, perhaps we should support it as a feature format, and then conversions to/from will be supported by the transformFeatures command. Create an issue if you feel so.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
Member

fnothaft commented Dec 21, 2016

@laserson

This comment has been minimized.

Show comment
Hide comment
@laserson

laserson Dec 21, 2016

Contributor

Nah, I've determined the path of maximum laziness is to just use another tool rather than revive the tool in ADAM 😛

Contributor

laserson commented Dec 21, 2016

Nah, I've determined the path of maximum laziness is to just use another tool rather than revive the tool in ADAM 😛

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Dec 21, 2016

Member

SGTM!

Member

fnothaft commented Dec 21, 2016

SGTM!

@fnothaft fnothaft deleted the fnothaft:doc-cli branch Dec 21, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment