Merge pull request #21 from flindersuni/develop

Develop to Master

docs/source/Access/GettingAccess.md

@@ -14,15 +14,15 @@ This page will guide you on the steps to get access and then connect to the HPC
 
 Currently, the HPC is not open to Undergraduates on a general basis. Exceptions are made on a case-by-case basis - please talk to your project supervisor first and then contact the HPC Support Team via [email](mailto:deepthought@flinders.edu.au).
 
-### Unix/Linux/MacOS
+## Unix/Linux/MacOS
 
 MacOS / MacOSX shares a similar procedure to Unix/BSD Based system. Unix/Linux & MacOS systems have native support for the SSH Protocol, used to connect to the HPC.
 
-#### The Windows Sub-System for Linux
+### The Windows Sub-System for Linux
 
 The windows Subsystem for Linux (WSL) allows you to run a Linux Distribution as a sub-system in windows. When following these instructions, a 'terminal' is the same as starting your WSL Distribution. Generally, if you are using the WSL, then following the Unix/Linux instructions.
 
-#### Getting Connected
+### Getting Connected
 
 The simplest manner is to open up a terminal window and type in the following, substituting FAN for your FAN as needed.
 
@@ -32,30 +32,30 @@ If all is OK it will ask for your password. Enter the same password you use to l
 
 ![](../_static/shellPasswordPromtImage.png)
 
-##### Success
+#### Success
 
 Upon a successful login, you should get a screen similar to this:
 
 ![](../_static/loginOKImage.png)
 
 If so, you are now connected and ready to start using the HPC!
 
-#### SSH Keys
+##### SSH Keys
 
 If you wish to setup password-less login via SSH Keys, you may do so.
 
-### Windows
+## Windows
 
 To connect to Deep Thought a SSH application such as PuTTy is required. Below is a short list of the possible programs you can use as a client to connect to the HPC. This guide will focus on Putty - but will be equally applicable to the other programs.
 
-#### Client Options
+### Client Options
 
 - [Putty](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html)
 - [KiTTY](http://www.9bis.net/kitty/#!pages/download.md)
 - [BitVise Client](https://www.bitvise.com/download-area)
 - [MobaXterm](https://mobaxterm.mobatek.net/)
 
-#### Getting Connected on Windows
+### Getting Connected on Windows
 
 Open PuTTy, and you are presented with this screen:
 
@@ -66,7 +66,7 @@ Open PuTTy, and you are presented with this screen:
 - Set the Port Number to 22
 - Click Open
 
-#### Logging In
+### Logging In
 
 If all has gone well, you will be presented with this screen:
 
@@ -77,14 +77,14 @@ If all has gone well, you will be presented with this screen:
 
 These are the same credentials you use to login to OKTA.
 
-#### Successful Login
+### Successful Login
 
 Upon a successful login, you should get a screen similar to this:
 
 ![](../_static/loginOKImage.png)
 
 If so, you are now connected and ready to start using the HPC.
 
-#### SSH Keys
+#### Windows SSH Keys
 
 As with the Unix/Linux/MacOS system, you may also setup SSH Keys for password-less logins. Be sure to follow the specific instructions for your client, as they will differ.

docs/source/FileTransfers/FileTransfersIntro.md

@@ -10,27 +10,26 @@ All file-transfers are done via Secure File Transfer Protocol (SFTP), or Secure
 
 ## Before we get started
 
-The HPC is a little different that your desktop at home when it comes to storage, not just computing power. It's a shared resource, so we cant store everybody's data for all time - there just isn't enough space. 
+The HPC is a little different that your desktop at home when it comes to storage, not just computing power. It's a shared resource, so we cant store everybody's data for all time - there just isn't enough space.
 
-On DeepThought, are two main storage tiers, with a smaller pool for your documents and scripts. Firstly our bulk storage (approx 250TB) is the 'Scratch' area (located at /scratch/user/<fan>) - and is slower, spinning Hard-Disk Drives (HDD's). The smaller, hyper-fast NVMe Solid-State Drives (located at /local) are roughly 400GB on the 'standard' nodes (1-16) and 1.5TB on the 'high-capacity' nodes (19-21).
+On DeepThought, are two main storage tiers, with a smaller pool for your documents and scripts. Firstly our bulk storage (approx 250TB) is the 'Scratch' area (located at /scratch/user/$FAN) - and is slower, spinning Hard-Disk Drives (HDD's). The smaller, hyper-fast NVMe Solid-State Drives (located at /local) are roughly 400GB on the 'standard' nodes (1-16) and 1.5TB on the 'high-capacity' nodes (19-21).
 
-There is a critical difference between these two locations. The /scratch area is a common storage area. You can access it from all of the login, management and compute nodes on the HPC. This is not the same as /local, which is only available on each compute node.  That is - if you job is running on Node001, the /local only exists on that particular node - you cannot access it anywhere else on the HPC. 
+There is a critical difference between these two locations. The /scratch area is a common storage area. You can access it from all of the login, management and compute nodes on the HPC. This is not the same as /local, which is only available on each compute node.  That is - if you job is running on Node001, the /local only exists on that particular node - you cannot access it anywhere else on the HPC.
 
 - /home/$FAN
 - /scratch/$FAN
 
-There are also two location that treated specially: 
+There following location is treated specially as it holds the older eRSA data and is slowly being phased out.
 
-- /mnt/
-- /r_drive/ 
+- /r_drive/
 
-Both of these locations will surface research specific mount points on request. If you have access to these locations, they auto-mount **on usage**. That is, simply try and touch or access the correct location that you have access to, and it will be handled automatically for you. For example, if you have access to a mount point called 'molecular_data', then the following command with surface that mount point to you - `cd /r_drive/molecular_data/`. 
+This location will surface specific mount points on request. If you have access to these locations, they auto-mount **on usage**. That is, simply try and touch or access the correct location that you have access to, and it will be handled automatically for you. For example, if you have access to a mount point called 'molecular_data', then the following command with surface that mount point to you - `cd /r_drive/molecular_data/`.
 
 ### /Home
 
 Your 'home' directories. This is a small amount of storage (~11TB total) to store your small bits and pieces. This is the analogous to the Windows 'Documents' folder.
 
-At a command promp, your home directory usually gets shortened to ~/. 
+At a command promp, your home directory usually gets shortened to ~/.
 
 #### What to store in /home
 
@@ -42,7 +41,7 @@ Here is a rough guide as to what should live in your /home/$FAN directory. In ge
 
 ### /Scratch
 
-Scratch is your working space. Depending upon your dataset, you may need to run your job here - this is not optimal and will be much slower than running it from /local. Scratch is still not an area to store your data permanently - there are no backups in place for the HPC, so ensure you follow the [HPC Research Data Flow ]() and the [HPC Job Data Flow]().
+Scratch is your working space. Depending upon your dataset, you may need to run your job here - this is not optimal and will be much slower than running it from /local. Scratch is still not an area to store your data permanently - there are no backups in place for the HPC, so ensure you follow the [HPC Research Data Flow]() and the [HPC Job Data Flow]().
 
 #### What to store in /scratch
 
@@ -51,45 +50,41 @@ Here is a rough guide as to what should live in your /scratch/$FAN directory. In
 - Job Working Data-sets
 - Intermediate files
 
-## Transferring Files
-
-All file-transfers are done via Secure File Transfer Protocol (SFTP) or Secure Copy Protocol (SCP). As was the same with the 'Getting Access', pick your platform:
-
-### Linux/Unix File Transfers
+## Linux/Unix File Transfers
 
-Linux / Unix based systems share native support for the SFTP Protocol. The Secure Copy Protocol (SCP) is also widely accepted, which can sometimes offer an edge in transfer speed.
+Linux / Unix based systems share native support for the SFTP Protocol. The Secure Copy Protocol (SCP) is also widely accepted, which can sometimes offer an edge in transfer speed. Tools such as RSYNC are also usable.
 
-#### The Windows Sub-System for Linux
+### The Windows Sub-System for Linux
 
 Since Windows 10 and Windows Server 2019, the windows Subsystem for Linux (WSL) allows you to run a Linux Distribution as a sub-system in windows. When following these instructions, a 'terminal' is the same as starting your WSL Distribution.
 
-#### Transferring Files to the HPC
+### Transferring Files to the HPC
 
 When using a *NIX based system, using the terminal is the fastest way to upload files to the HPC.
 
-#### The Quick Version
+### The Quick Version
 
 Substitute your filename, FAN and Password, type scp FILENAME FAN@deepthought.flinders.edu.au:/home/FAN then hit enter.
 Enter your password when prompted. This will put the file in your home directory on DeepThought. It looks (when substituted accordingly) similar to:
 
 ![](../_static/SCPExampleImage.png)
 
-#### The Longer Version
+### The Longer Version
 
 To download files from DeepThought, you simply need to invert that command to point to either:
 
 - A name of a Computer that Deepthough 'knows' about.
 - An IP Address that Deepthought can reach.
 
-#### Transfers By Computer Name
+### Transfers By Computer Name
 
 If you know the hostname of the computer, you can substitute this to transfer files back to your machine. The command stays the same, mostly. You still follow the same idea, we just change where we are pointing it. This one assumed you are transferring it to a Linux/Unix based machine.
 
 The command will take this form:
 
 ![](../_static/SCPByHostname.png)
 
-#### Transfer By IP Address
+### Transfer By IP Address
 
 If you don't know your computer IP, then the commands of:
 
@@ -100,15 +95,15 @@ Will be your friend to figure out what it is. Just like above, we slightly chang
 
 ![](../_static/SCPByIP.png)
 
-### Windows
+## Windows
 
 Windows doesn't support the SFTP protocol in a native way. Thankfully, there are lots of clients written to do just this for us.
 
-#### Sub-System for Linux
+### Sub-System for Linux
 
 You can use the WSL for this - head on over to the [Linux](#TransferringFiles) Guide.
 
-#### Potential Client List
+### Potential Client List
 
 This is not an exhaustive list - feel free to use whatever you wish that supports the SFTP protocol.
 
@@ -117,7 +112,7 @@ This is not an exhaustive list - feel free to use whatever you wish that support
 
 This guide will focus on WinSCP.
 
-#### Getting Connected with WinSCP
+### Getting Connected with WinSCP
 
 Open WinSCP, enter deepthought.flinders.edu.au as the host to connect to, and click Login. You should have a screen that looks like this.
 

docs/source/LinuxCommands/LinuxIntro.md

@@ -1,4 +1,4 @@
-# Linux Commands: HPC Style
+# Linux Command Line Interface
 
 We try not to re-invent the wheel, and there is already a wonderful guide for basic linux commands, written by the excellent people at Harvard. Head on over to [here](https://informatics.fas.harvard.edu/basic-unix-workshop.html) and have a read!
 

docs/source/flindershpc2021-ris.ris

@@ -0,0 +1,10 @@
+TY  - GEN
+TI  - Deep Thought (HPC)
+AU  - Flinders University
+DA  - 2021///
+PY  - 2021
+PB  - Flinders University
+UR  - https://deepthoughtdocs.flinders.edu.au/en/latest/
+KW  - High performance computing
+KW  - HPC
+ER  - 

docs/source/index.rst

@@ -6,47 +6,81 @@
 Welcome to the DeepThought Documentation
 =========================================
 
-The new Flinders High Performance Computing (HPC) solution is called Deep Thought.  
-Comprising of high-end DELL hardware and services, DeepThought delivers a core enterprise HPC service, where moderate usage is delivered at no cost to researchers or colleges.
+The new Flinder HPC is is called DeepThought. This new HPC comprises of AMD EPYC based hardware and next-generation management software, allowing for a dynamic and agile HPC service. 
 
 .. attention::
     This documentation is under active development, meaning that it can
     change over time as we improve it. Please email deepthought@flinders.edu.au if
     you require assistance. We also welcome user contributions  to this documentation 
     - contact the same email above so we can grant you access.
 
-.. toctree::
-   :maxdepth: 2
-   :caption: User Documentation
-
-   Access/GettingAccess.md
-   Storage/storageusage.rst
-   dataflow/hpcresearchdataflow.rst 
-   FileTransfers/FileTransfersIntro.md
-   LinuxCommands/LinuxIntro.md 
-   dataflow/hpcjobdataflow.rst
-   SLURM/SLURMIntro.md
-   ModuleSystem/LMod.md
 
-.. toctree::
-   :maxdepth: 2
-   :caption: Technical Specifications
+Attribution
+----------------------
 
-   system/deepthoughspecifications.md
+If you use the HPC to form a part of your research, you should attribute your usage. 
+Flinders has minted a DOI that points to this documentation, specific for the HPC Service. It will also allow for tracking the research outputs that the HPC has contributed to.  
 
-.. toctree::
-   :maxdepth: 2
-   :caption: FAQ & Known Issues
+Text Citation
+++++++++++++++
+
+.. _ARDC Data Citation: https://ardc.edu.au/resources/working-with-data/citation-identifiers/data-citation/
+
+The below text citation follows the standard Australian Research Data Commons (ARDC) format for attributing data and software. For more information on this type of attribution, 
+visit the `ARDC Data Citation`_ page. 
 
-   FAQ/faq.rst
-   FAQ/knownissues.rst
+``Flinders University (2021). DeepThought (HPC). Retrieved from https://doi.org/10.25957/FLINDERS.HPC.DEEPTHOUGHT``
 
 
+Reference Managers
++++++++++++++++++++
+
+The following files are provided for integration into your reference manager of choice. To save the file, 'Right Click -> Save Link As..'. 
+
+.. _BibTex: https://raw.githubusercontent.com/flindersuni/DeepThoughtHPC-docs/master/docs/source/flindershpc2021-bibtex.bib
+.. _EndNote: https://raw.githubusercontent.com/flindersuni/DeepThoughtHPC-docs/master/docs/source/flindershpc2021-endnote.xml
+.. _RIS: https://raw.githubusercontent.com/flindersuni/DeepThoughtHPC-docs/master/docs/source/flindershpc2021-ris.ris
+
+- BibTex_ 
+- EndNote_
+- RIS_
+
+
+Table of Contents
+====================
+
+.. toctree::
+    :maxdepth: 1
+    :caption: User Documentation
+
+    Access/GettingAccess.md
+    Storage/storageusage.rst
+    dataflow/hpcresearchdataflow.rst
+    FileTransfers/FileTransfersIntro.md
+    LinuxCommands/LinuxIntro.md
+    dataflow/hpcjobdataflow.rst
+    SLURM/SLURMIntro.md
+    ModuleSystem/LMod.md
+
+.. toctree::
+    :maxdepth: 1
+    :caption: Technical Specifications
+
+    system/deepthoughspecifications.md
+
+.. toctree::
+    :maxdepth: 1
+    :caption: FAQ & Known Issues
+
+    FAQ/faq.rst
+    FAQ/knownissues.rst
+
+
 .. toctree::
-   :maxdepth: 2
-   :caption: Software & Support Policies 
+   :maxdepth: 1
+   :caption: Software & Support Policies
 
-   policies/fairuse.rst 
+   policies/fairuse.rst
    policies/accessandpermissions.rst
 
 Citing DeepThoughtHPC
@@ -59,12 +93,14 @@ Flinders University (2021). DeepThought (HPC). Retrieved from https://doi.org/10
 .. _BibTex: https://raw.githubusercontent.com/flindersuni/DeepThoughtHPC-docs/master/docs/source/flindershpc2021-bibtex.bib
 .. _EndNote: https://raw.githubusercontent.com/flindersuni/DeepThoughtHPC-docs/master/docs/source/flindershpc2021-endnote.xml
 
+
+
 Acknowledgements
 ----------------
 
 We recognise the respect the trademarks of all third-party providers referenced in this documentation. Please see the respective EULAs for software packages used in configuring your own environment based on this knowledgebase.
 
 License
-^^^^^^^
+=========
 
 This documentation is released under the `Creative-Commons: Attribution-ShareAlike 4.0 International <http://creativecommons.org/licenses/by-sa/4.0/>`_ license.