diff --git a/.gitignore b/.gitignore index 279b4fc..34d53c1 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ env/ site/ +.venv/ \ No newline at end of file diff --git a/docs/using/community_examples.md b/docs/using/community_examples.md new file mode 100644 index 0000000..3aa2ad2 --- /dev/null +++ b/docs/using/community_examples.md @@ -0,0 +1,23 @@ +# Community Usage Examples + +This collects [github gists][github_gists] which HTCF users have written to +serve as examples of HTCF usage patterns. We encourage you to write your own +gists which describe common tasks you do on the cluster, and share them. This +is a good way of keeping a record of how you use HTCF for yourself and future +lab mates, and can help the whole HTCF community improve and optimize usage. +Feel free to ask questions of the gist writer through the github interface. + +- [Turning Exploratory Data Analysis into containerized cmd line scripts in R][eda_to_containerized_cmd] + + > This gist describes a method of interactively exploring data in R in a way that +allows the user to simultaneously develop a command line script. Critically, this +describes how to use [renv][renv_link] to manage a virtual environment. +This makes it simple to create a [docker][docker_link] container. Since +[singularity][singularity_link] can run docker containers, this means that you +can easily turn EDA into reproducible results which may be easily run on HTCF. + +[github_gists]: https://docs.github.com/en/get-started/writing-on-github/editing-and-sharing-content-with-gists/creating-gists +[renv_link]: https://rstudio.github.io/renv/articles/renv.html +[docker_link]: https://www.docker.com/ +[singularity_link]: https://sylabs.io/singularity/ +[eda_to_containerized_cmd]: https://gist.github.com/cmatKhan/e08a262b45bed5465301530b7319e6b3 diff --git a/docs/using/getstarted.md b/docs/using/getstarted.md index b0b5f3b..8b2db15 100644 --- a/docs/using/getstarted.md +++ b/docs/using/getstarted.md @@ -20,6 +20,8 @@ The login server, `login.htcf.wustl.edu` is accessible via ssh. Before using the HTCF, it's important to read through and understand: +1. The difference between [login, interactive and compute](#jobs) nodes + 1. The [HTCF Storage](../storage/index.md) 2. Using [software](../software.md) on the HTCF @@ -48,11 +50,20 @@ High Throughput Storage is a distrubuted file system able to handle tens of GBs/ ## Jobs -### Resources +When you first log in to HTCF, you are interacting with a part of the cluster +which is specifically designated for users to log in, log off, submit jobs and +move data between storage spaces (eg `scratch` and `lts`). -The number of CPUs and MBs of RAM per node can be found using the Slurm sinfo command: +!!! Note + Do not do any analysis or computation on the login node - $ sinfo -N -p general -o '%n %c %m' +If you need to prototype or do some interactive work (eg, exploratory data analysis) +on HTCF, then you should use an [interative session](#Interactive) + +The ultimate goal this type of interactive work is to automate tasks such that +they may be submitted to the SLURM scheduler and run on the `compute` nodes. +The majority of the compute resources on HTCF are these general compute nodes. +Scheduling on these nodes is achieved using the command `sbatch`. ### Interactive @@ -155,3 +166,10 @@ $ srun --ntasks-per-node=1 --jobid=12345 nvidia-smi | 0 12043 C guppy_basecaller 4753MiB | +-----------------------------------------------------------------------------+ ~~~~ + +### Resources + +The number of CPUs and MBs of RAM per node can be found using the Slurm sinfo command: + + $ sinfo -N -p general -o '%n %c %m' + \ No newline at end of file diff --git a/mkdocs.yml b/mkdocs.yml index c3e4b08..8bb7150 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -70,3 +70,4 @@ nav: - Examples: - 'software/examples/r_mpi/index.md' # - R+MPI: 'software/examples/r_with_mpi.md' +- "Use Examples": 'using/community_examples.md'