New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleaned up docs. #1642

Closed
wants to merge 19 commits into
base: master
from

Conversation

Projects
6 participants
@gunjanbaid
Contributor

gunjanbaid commented Jul 26, 2017

Cleaned up formatting, language, and broken links. @fnothaft Let me know if it's easier for me to open multiple PRs (one for each file?).

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jul 26, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 5f8d939 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

coveralls commented Jul 26, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 5f8d939 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 26, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2288/
Test PASSed.

AmplabJenkins commented Jul 26, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2288/
Test PASSed.

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Jul 27, 2017

Member

Thanks @gunjanbaid! A single PR is a-OK by me! I'll make a pass shortly.

Member

fnothaft commented Jul 27, 2017

Thanks @gunjanbaid! A single PR is a-OK by me! I'll make a pass shortly.

@fnothaft

Thanks @gunjanbaid! Looks solid!

I saw lots of places where we lost anchors/anchor links in the documentation. I'm guessing this was intentional? My preference is to keep the links, as they aid navigation in the HTML and PDF copies. What's your general line on this?

Show outdated Hide outdated docs/source/01_intro.md Outdated
Show outdated Hide outdated docs/source/02_installation.md Outdated
Show outdated Hide outdated docs/source/02_installation.md Outdated
Show outdated Hide outdated docs/source/02_installation.md Outdated
Show outdated Hide outdated docs/source/02_installation.md Outdated
Show outdated Hide outdated docs/source/40_deploying_ADAM.md Outdated
Show outdated Hide outdated docs/source/40_deploying_ADAM.md Outdated
Show outdated Hide outdated docs/source/40_deploying_ADAM.md Outdated
Show outdated Hide outdated docs/source/40_deploying_ADAM.md Outdated
Show outdated Hide outdated docs/source/40_deploying_ADAM.md Outdated
Shutdown the cluster using:
```
cgcloud terminate-cluster -c cluster1 spark
```
#### CGCoud options and Spot Instances
View help docs for all options of the the `cgcloud create-cluster` command:
View help docs for all options of the `cgcloud create-cluster` command:
```
cgcloud create-cluster -h

This comment has been minimized.

@gunjanbaid

gunjanbaid Jul 27, 2017

Contributor

Some code blocks with bash commands (like these) start with $, while some don't. Is there a preferred style? I can update so they are all consistent.

@gunjanbaid

gunjanbaid Jul 27, 2017

Contributor

Some code blocks with bash commands (like these) start with $, while some don't. Is there a preferred style? I can update so they are all consistent.

This comment has been minimized.

@heuermh

heuermh Jul 27, 2017

Member

For me, if the bash code block shows one or commands that the user would be likely to copy and paste into a bash shell, then I typically don't include the $.

mvn install
./bin/adam-submit transformAlignments foo.bam bar.adam
spark-submit conductor-0.5-SNAPSHOT-distribution.jar hdfs://bar.adam s3://bar.adam

If the code block is showing the output of a command, then I would include the $.

$ adam-submit --help
Using ADAM_MAIN=org.bdgenomics.adam.cli.ADAMMain
Using SPARK_SUBMIT=/usr/local/bin/spark-submit

       e         888~-_          e             e    e
      d8b        888   \        d8b           d8b  d8b
     /Y88b       888    |      /Y88b         d888bdY88b
    /  Y88b      888    |     /  Y88b       / Y88Y Y888b
   /____Y88b     888   /     /____Y88b     /   YY   Y888b
  /      Y88b    888_-~     /      Y88b   /          Y888b

Usage: adam-submit [<spark-args> --] <adam-args>
...
@heuermh

heuermh Jul 27, 2017

Member

For me, if the bash code block shows one or commands that the user would be likely to copy and paste into a bash shell, then I typically don't include the $.

mvn install
./bin/adam-submit transformAlignments foo.bam bar.adam
spark-submit conductor-0.5-SNAPSHOT-distribution.jar hdfs://bar.adam s3://bar.adam

If the code block is showing the output of a command, then I would include the $.

$ adam-submit --help
Using ADAM_MAIN=org.bdgenomics.adam.cli.ADAMMain
Using SPARK_SUBMIT=/usr/local/bin/spark-submit

       e         888~-_          e             e    e
      d8b        888   \        d8b           d8b  d8b
     /Y88b       888    |      /Y88b         d888bdY88b
    /  Y88b      888    |     /  Y88b       / Y88Y Y888b
   /____Y88b     888   /     /____Y88b     /   YY   Y888b
  /      Y88b    888_-~     /      Y88b   /          Y888b

Usage: adam-submit [<spark-args> --] <adam-args>
...
@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 3329607 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

coveralls commented Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 3329607 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2292/
Test PASSed.

AmplabJenkins commented Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2292/
Test PASSed.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7643f64 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

coveralls commented Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7643f64 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2293/
Test PASSed.

AmplabJenkins commented Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2293/
Test PASSed.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

coveralls commented Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2294/
Test PASSed.

AmplabJenkins commented Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2294/
Test PASSed.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

coveralls commented Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2295/
Test PASSed.

AmplabJenkins commented Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2295/
Test PASSed.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

coveralls commented Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2296/
Test PASSed.

AmplabJenkins commented Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2296/
Test PASSed.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

coveralls commented Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2297/
Test PASSed.

AmplabJenkins commented Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2297/
Test PASSed.

@coveralls

This comment has been minimized.

Show comment
Hide comment
@coveralls

coveralls Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

coveralls commented Jul 27, 2017

Coverage Status

Coverage remained the same at 83.961% when pulling 7cb8ea5 on gunjanbaid:docs into c8a2202 on bigdatagenomics:master.

@AmplabJenkins

This comment has been minimized.

Show comment
Hide comment
@AmplabJenkins

AmplabJenkins Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2298/
Test PASSed.

AmplabJenkins commented Jul 27, 2017

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/ADAM-prb/2298/
Test PASSed.

@fnothaft

LGTM! Thanks @gunjanbaid!

From ADAM shell, or as parameter to ADAM submit, you would refer HDFS URLs
such as:
From the ADAM shell, or as a parameter to ADAM submit, you would refer to HDFS URLs like this:

This comment has been minimized.

@fnothaft

fnothaft Aug 2, 2017

Member

Note to self: break this line at 80 characters when merging.

@fnothaft

fnothaft Aug 2, 2017

Member

Note to self: break this line at 80 characters when merging.

on your local machine point your web-browser to http://ip_of_spark_master:4040/
8080 on `spark-master`, go to Security Groups in the AWS console
and open inbound TCP for those ports from your local IP address. Find the
IP address of `spark-master`, which is part of the Linux command prompt. On your local machine,

This comment has been minimized.

@fnothaft

fnothaft Aug 2, 2017

Member

Note to self: break this line at 80 characters when merging.

@fnothaft

fnothaft Aug 2, 2017

Member

Note to self: break this line at 80 characters when merging.

@@ -520,8 +521,8 @@ For those groups with access to a HPC cluster with [Slurm](https://en.wikipedia.
managing a number of compute nodes with local and/or network attached storage, it is possible to spin up a
temporary Spark cluster for use by ADAM.
While the full IO bandwidth benefits of Spark processing are likely best realized through a set of
co-located compute/storage nodes, depending on your network setup you may find Spark deployed on HPC
The full IO bandwidth benefits of Spark processing are likely best realized through a set of

This comment has been minimized.

@fnothaft

fnothaft Aug 2, 2017

Member

Note to self: break this line at 80 characters when merging.

@fnothaft

fnothaft Aug 2, 2017

Member

Note to self: break this line at 80 characters when merging.

We'd love to hear feedback on your experience running ADAM on HPC/Slurm or other deployment architectures,
and let us know of any problems you run into via the mailing list or Gitter.
We would love to hear feedback on your experience running ADAM on HPC/Slurm or other deployment architectures.

This comment has been minimized.

@fnothaft

fnothaft Aug 2, 2017

Member

Note to self: break this line at 80 characters when merging.

@fnothaft

fnothaft Aug 2, 2017

Member

Note to self: break this line at 80 characters when merging.

@heuermh

heuermh approved these changes Aug 2, 2017

@fnothaft

This comment has been minimized.

Show comment
Hide comment
@fnothaft

fnothaft Aug 2, 2017

Member

Thanks @gunjanbaid! I've merged this manually as 6573997 and 9032698.

Member

fnothaft commented Aug 2, 2017

Thanks @gunjanbaid! I've merged this manually as 6573997 and 9032698.

@fnothaft fnothaft closed this Aug 2, 2017

@heuermh heuermh added this to the 0.23.0 milestone Dec 7, 2017

@heuermh heuermh added this to Completed in Release 0.23.0 Jan 4, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment