Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
env/
site/
.venv/
23 changes: 23 additions & 0 deletions docs/using/community_examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# Community Usage Examples

This collects [github gists][github_gists] which HTCF users have written to
serve as examples of HTCF usage patterns. We encourage you to write your own
gists which describe common tasks you do on the cluster, and share them. This
is a good way of keeping a record of how you use HTCF for yourself and future
lab mates, and can help the whole HTCF community improve and optimize usage.
Feel free to ask questions of the gist writer through the github interface.

- [Turning Exploratory Data Analysis into containerized cmd line scripts in R][eda_to_containerized_cmd]

> This gist describes a method of interactively exploring data in R in a way that
allows the user to simultaneously develop a command line script. Critically, this
describes how to use [renv][renv_link] to manage a virtual environment.
This makes it simple to create a [docker][docker_link] container. Since
[singularity][singularity_link] can run docker containers, this means that you
can easily turn EDA into reproducible results which may be easily run on HTCF.

[github_gists]: https://docs.github.com/en/get-started/writing-on-github/editing-and-sharing-content-with-gists/creating-gists
[renv_link]: https://rstudio.github.io/renv/articles/renv.html
[docker_link]: https://www.docker.com/
[singularity_link]: https://sylabs.io/singularity/
[eda_to_containerized_cmd]: https://gist.github.com/cmatKhan/e08a262b45bed5465301530b7319e6b3
24 changes: 21 additions & 3 deletions docs/using/getstarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ The login server, `login.htcf.wustl.edu` is accessible via ssh.

Before using the HTCF, it's important to read through and understand:

1. The difference between [login, interactive and compute](#jobs) nodes

1. The [HTCF Storage](../storage/index.md)

2. Using [software](../software.md) on the HTCF
Expand Down Expand Up @@ -48,11 +50,20 @@ High Throughput Storage is a distrubuted file system able to handle tens of GBs/

## Jobs

### Resources
When you first log in to HTCF, you are interacting with a part of the cluster
which is specifically designated for users to log in, log off, submit jobs and
move data between storage spaces (eg `scratch` and `lts`).

The number of CPUs and MBs of RAM per node can be found using the Slurm sinfo command:
!!! Note
Do not do any analysis or computation on the login node

$ sinfo -N -p general -o '%n %c %m'
If you need to prototype or do some interactive work (eg, exploratory data analysis)
on HTCF, then you should use an [interative session](#Interactive)

The ultimate goal this type of interactive work is to automate tasks such that
they may be submitted to the SLURM scheduler and run on the `compute` nodes.
The majority of the compute resources on HTCF are these general compute nodes.
Scheduling on these nodes is achieved using the command `sbatch`.

### Interactive

Expand Down Expand Up @@ -155,3 +166,10 @@ $ srun --ntasks-per-node=1 --jobid=12345 nvidia-smi
| 0 12043 C guppy_basecaller 4753MiB |
+-----------------------------------------------------------------------------+
~~~~

### Resources

The number of CPUs and MBs of RAM per node can be found using the Slurm sinfo command:

$ sinfo -N -p general -o '%n %c %m'

1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -70,3 +70,4 @@ nav:
- Examples:
- 'software/examples/r_mpi/index.md'
# - R+MPI: 'software/examples/r_with_mpi.md'
- "Use Examples": 'using/community_examples.md'