Merge pull request #14 from flindersuni/develop

Develop to Master Update

docs/source/FAQ/faq.rst

@@ -6,17 +6,18 @@ Below are some of the common steps that the team has been asked to resolve more
 
 What are the SLURM Partitions? 
 ===============================
-There are just two (for now): 
+There are three at this point: 
 
 * hpc_general 
+* hpc_gpu
 * hpc_melfeu 
 
 You can omit the 
 
 * #SBATCH partition=<name> directive
 
 
-as the sane-default for you is the hpc_general partition. 
+as the sane-default for you is the hpc_general partition. If you need access to the GPU's you **must** user the hpc_gpu queue.
 
 SLURM - Tasks & OpenMPI/MPI
 ===========================
@@ -129,4 +130,11 @@ Things like:
 * Preparing Scripts
 
 As a good rule, if it takes more than 10-15 minutes or > 2GB RAM, it should be run a SLURM Job, not on the head nodes. 
-Anything that uses too many resources on the head nodes will be *Terminated* **WITHOUT WARNING**.
+Anything that uses too many resources on the head nodes will be *Terminated* **WITHOUT WARNING**.
+
+
+My Mount Point at /r_drive or /mnt is missing! 
+===============================================
+Anything under those directories above are set to transparent auto-mount. 
+Try and move to a directory or touch a file, and the operating system will mount and surface the locations for you 
+automatically. 

docs/source/FileTransfers/FileTransfersIntro.md

@@ -6,11 +6,18 @@ Transferring files to the HPC will change depending upon your OS. Thankfully, th
 
 The HPC is a little different that your desktop at home when it comes to storage (not just computing power!). It's a shared resource, so we cant store everybody's data for all time - there just isn't enough space!
 
-So, we have two main storage locations we use; Linux Guru's some of this is old hat - please keep reading, it'll get to new things soon!
+So, we have two main storage locations we use which are:
 
 - /home/$FAN
 - /scratch/$FAN
 
+There are also two location that treated specially: 
+
+- /mnt/
+- /r_drive/ 
+
+Both of these locations will surface research specific mount points on request. If you have access to these locations, they auto-mount **on usage**. That is, simply try and touch or access the correct location that you have access to, and it will be handled automatically for you.
+
 ### /Home
 
 Your 'home' directories. This is a small amount of storage to store your small bits and pieces. This is the analogous to the Windows 'Documents' folder.

docs/source/ModuleSystem/LMod.md

@@ -6,9 +6,29 @@ DeepThought HPC uses the [LMod](https://lmod.readthedocs.io/en/latest/) (Load MO
 
 Best way to think of Module is a singular program version + all of its associated dependencies to run correctly.
 
-## Writing Your Own Modules
+## How Do I Install Software? 
+There are multiple ways to install software on the HPC. Below is an expansion on some of the common ones. The short and sweet version is that, if you compile/install it yourself to your /home or a Virtual Environment of some kind, you are free to do whatever you want! 
+
+The HPC support team will need to action your request if you need something big and complicated like ANSYS, GNU Octave, a new version of R or other similar large and complicated programs. 
+
+### Python / Conda
+The HPC Team allows you to install your own packages by using the inbuilt package manager tools, like Pythons 'pip', or Conda. 
+
+As an example, you can create a Conda Virtual Environment - this is under your complete control and you may install, remove or alter it as you wish. This is also the same for Pythons 'venv', which functions in much the same way.  
+
+The Conda Guide is located at: [Conda Guide](https://docs.conda.io/projects/conda/en/latest/user-guide/getting-started.html#managing-envs)
+
+The Python Guide is located at: [Python Guide](https://packaging.python.org/guides/installing-using-pip-and-virtual-environments/#creating-a-virtual-environment)
+
+### Compile Your Own 
+The HPC uses the FOSS Toolchain, as detailed in the [Fair Usage](../policies/fairuse.html#toolchains) Policy.  Should you wish to compile and use your own software, simply load the associated module (eg, foss-2020a) which will load up the associated GCC Toolchains and libraries. 
 
-You can [write your own module files](https://lmod.readthedocs.io/en/latest/015_writing_modules.html#) if you want! This is supported on a best-effort basis by the HPC support team.
+#### My Toolchain isn't Listed
+Should you require a different Toolchain, like LLVM or Go and it is not listed under the `module avail` list, you can either: 
+
+1.) Bootstrap the compiler + libraries yourself in your /home directory, managing it and any programs yourself
+
+2.) Contact the HPC Support Team, either via [Email](mailto:deepthought@flinders.edu.au) or ServiceOne
 
 ## Module Format
 
@@ -79,9 +99,12 @@ Generally speaking, we can install almost all Linux/CentOS bounded software/appl
 
 Otherwise, there is nothing stopping you installing the program locally for yourself! If you run into issues installing software then open an ServiceOne ticket and again, Digital Research Services will help on a best-effort basis.
 
+___
+
 ### An Important Note
 
 The software must in all cases be appropriately licensed.
+___
 
 ## Currently Installed Modules
 
@@ -94,3 +117,7 @@ This is the list of software that has been 'hand rolled' as it contains either t
 ### Additional Software
 
 There are additional software collections will be made available in a near future. 
+
+## Writing Your Own Modules
+
+You can [write your own module files](https://lmod.readthedocs.io/en/latest/015_writing_modules.html#) if you want! This is supported on a best-effort basis by the HPC support team.

docs/source/policies/accessandpermissions.rst

@@ -32,7 +32,6 @@ General Cluster Rules
 
 5) Do not access the compute nodes directly.
 
-6) Utlise /local on the compute nodes for your data sets if possible.
 
 
 Permissions & Access Levels 
@@ -73,7 +72,9 @@ and, while the Support Team is asked for these permissions quite often, the pote
 
 3) Elevated Access to the HPC System 
 
-4) Access to Managerial Systems 
+4) Access to the Cluster Management System
+
+5) Access to the Storage Analytics 
 
 
 If You Break These Rules 

docs/source/system/deepthoughspecifications.md

@@ -8,7 +8,7 @@ The SLURM Scheduler as the notion of 'Job Queue' or 'Partitions'.  These manage
 
 |Partition Name |   Number of Nodes |   Usage / Purpose                    | Max Runtime    |
 |---------------|   -------         |   ------                             | -----          |
-|hpc_general    |   13              | General Usage Pool                   | UNLIMITED      |
+|hpc_general    |   15              | General Usage Pool                   | UNLIMITED      |
 |hpc_melfeu     |   2               | Molecular Biology Lab private Nodes. | UNLIMITED      |  
 
 ## Storage Layout
@@ -17,9 +17,9 @@ Scratch: ~80TB of scratch disk, mounted on all nodes
 
 ## Node Breakdown
 
-- 17 Compute Nodes, totalling 1376 Cores and 6.28TB of RAM combined.
+- 17 Compute Nodes, with ~180 Cores and ~10TB of RAM total
 - 2 Login Nodes, with High-Availability Failover
-- 4 V100 Nvidia TESLA GPU's with 32GB VRAM
+- 4 V100 Nvidia TESLA GPU's with 32GB VRAM per GPU
 
 ### General Nodes
 
@@ -43,7 +43,7 @@ There are 2 dedicated GPU nodes, each with:
 
 ### High Capacity Node
 
-There is a single High-Capacity node with:
+There is are 3 High-Capacity nodes with:
 
 - CPU:
   - 2 x AMD EPYC 7742 @2.25Ghz with 64 Cores / 128 Threads