Merge pull request #4 from flindersuni/develop

Develop into Master, Docs V0.1 Update

.readthedocs.yml

@@ -17,7 +17,7 @@ sphinx:
 formats: all
 
 # Optionally set the version of Python and requirements required to build your docs
-#python:
-#  version: 3.7
-#  install:
-#    - requirements: docs/requirements.txt
+python:
+  version: 3.7
+  install:
+    - requirements: docs/requirements.txt

docs/requirements.txt

@@ -0,0 +1,12 @@
+sphinx_markdown_tables 
+Pygments==2.3.1 
+setuptools==41.0.1 
+docutils==0.14 
+mock==1.0.1 
+pillow>=5.4.1 
+alabaster>=0.7,<0.8,!=0.7.5 
+commonmark==0.8.1 
+recommonmark==0.5.0 
+sphinx<2 
+sphinx-rtd-theme<0.5 
+readthedocs-sphinx-ext<1.1

docs/source/Access/GettingAccess.md

@@ -0,0 +1,89 @@
+# Getting Access to DeepThought
+
+This page will guide you on the steps to get access and then connect to the HPC on your operating system of choice.
+
+## Flinders Staff / HDR Students
+
+1. Fire of a ServiceOne Ticket asking for Access to the HPC
+
+2. Pick your Operating System ([Unix/Linux/MacOS](#unix-linux-macos) or [Windows](#windows) )
+
+3. Read up on some basic [SLURM](../SLURM/SLURMIntro.md)
+
+### Unix/Linux/MacOS
+
+MacOS / MacOSX shares a similar procedure to Unix/BSD Based system. Unix/Linux & MacOS systems have native support for the SSH Protocol, used to connect to the HPC.
+
+#### The Windows Sub-System for Linux
+
+The windows Subsystem for Linux (WSL) allows you to run a Linux Distribution as a sub-system in windows. When following these instructions, a 'terminal' is the same as starting your WSL Distribution.
+
+#### Getting Connected
+
+The simplest manner is to open up a terminal window and type in the following, substituting FAN for your FAN as needed.
+
+- ssh FAN@deepthought.flinders.edu.au
+
+If all is OK it will ask for your password. Enter the same password you use to login to OKTA.
+
+![SSHPasswordPrompt](../../_static/shellPasswordPromtImage.png)
+
+##### Success
+
+Upon a successful login, you should get a screen similar to this:
+
+![LoginOK](../../_static/loginOKImage.png)
+
+If so, you are now connected and ready to start using the HPC!
+
+#### SSH Keys
+
+If you wish to setup password-less login via SSH Keys, you may do so.
+
+### Windows
+
+To connect to Deep Thought a SSH application such as PuTTy is required. Below is a short list of the possible programs you can use as a client to connect to the HPC. This guide will focus on Putty - but will be equally applicable to the other programs.
+
+#### The Windows Sub-System for Linux
+
+If you are using the Windows SubSystem for Linux (WSL), then go ahead and read the [Unix/Linux](#unix-linux-macos) instructions instead.
+
+#### Client Options
+
+- [Putty](https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html)
+- [KiTTY](http://www.9bis.net/kitty/#!pages/download.md)
+- [BitVise Client](https://www.bitvise.com/download-area)
+
+#### Getting Connected
+
+Open PuTTy, and you are presented with this screen:
+
+![](../../_static/puttyAccessImage.png)
+
+- Fill in the hostname to be: deepthought.flinders.edu.au,
+- Change the Connection Type to SSH
+- Set the Port Number to 22
+- Click Open
+
+#### Logging In
+
+If all has gone well, you will be presented with this screen:
+
+![](../../_static/puttyLoginImage.png)
+
+- Your Username is your FAN
+- Your Password is your FAN Password.
+
+These are the same credentials you use to login to OKTA.
+
+#### Success
+
+Upon a successful login, you should get a screen similar to this:
+
+![](../../_static/loginOKImage.png)
+
+If so, you are now connected and ready to start using the HPC!
+
+#### SSH Keys 
+
+As with the Unix/Linux/MacOS system, you may also setup SSH Keys for password-less logins. Be sure to follow the specific instructions for your client, as they will differ! 

docs/source/FAQ/faqandissues.rst

@@ -0,0 +1,29 @@
+*****
+FAQ 
+*****
+
+The always present 'We get asked these a lot' section. Currently, its empty!
+
+
+******************
+Known Issues
+******************
+
+Below are the 'known issues' with the documentation at this point.  This is not an issue tracker for the HPC, so please don't try and lodge an issue with Deepthought here!
+
+PDF Version
+==============
+
+Images are missing in some sections on the PDF Version 
+Alt-Text  appears when it should not in the PDF Version 
+
+EBPUB Version 
+================
+
+EPUB Version is missing some pages of content
+
+
+Web / ReadTheDocs / Online Version 
+====================================
+
+Occasionally, the Sidebar Mis-behaves when updating the Table of Contents 

docs/source/FileTransfers/FileTransfersIntro.md

@@ -0,0 +1,119 @@
+# Transferring Files to Deepthought
+
+Transferring files to the HPC will change depending upon your OS. Thankfully, there are some excellent tools that take this from 'potentially-scary' to 'click a button or two'.
+
+## Before we get started
+
+The HPC is a little different that your desktop at home when it comes to storage (not just computing power!). It's a shared resource, so we cant store everybody's data for all time - there just isn't enough space!
+
+So, we have two main storage locations we use; Linux Guru's some of this is old hat - please keep reading, it'll get to new things soon!
+
+- /home/$FAN
+- /scratch/$FAN
+
+### /Home
+
+Your 'home' directories. This is a small amount of storage to store your small bits and pieces. This is the analogous to the Windows 'Documents' folder.
+
+At a command promp, your home directory usually gets shortened to ~/ - we will cover this more in the Linux Guide.
+
+#### What to store in /home
+
+Here is a rough guide as to what should live in your /home/$FAN directory. In general, you want small, little things is here.
+
+- SLURM Scripts
+- Results from Jobs.
+- 'Small' Data-Sets (<1GB)
+
+### /Scratch
+
+Scratch is your working space. Whenever you are running a job, it should be running in the the /scratch area. This storage area is quicker and isolated from the others - meaning that even under high usage you are not going to slow down much (if at all) waiting to read data to or from the disks.
+
+It's also much, much larger than your /home area. Some of the working data-sets we have seen on Deep Thought have gotten out to Multiple Terabytes! Thankfully, /scratch can handle this, however /home would not.
+
+#### What to store in /scratch
+
+Here is a rough guide as to what should live in your /scratch/$FAN directory. In general, anything large, bulky and only needed for a little while should go here.
+
+- Job Working Data-sets
+- Intermediate files
+
+## Transferring Files
+
+All file-transfers are done via Secure File Transfer Protocol (SFTP). As was the same with the 'Getting Access', pick your platform:
+
+### Linux/Unix File Transfers
+
+Linux / Unix based systems share native support for the SFTP Protocol. The Secure Copy Protocol (SCP) is also widely accepted, which can sometimes offer an edge in transfer speed.
+
+#### The Windows Sub-System for Linux
+
+Since Windows 10 and Windows Server 2019, the windows Subsystem for Linux (WSL) allows you to run a Linux Distribution as a sub-system in windows. When following these instructions, a 'terminal' is the same as starting your WSL Distribution.
+
+#### Transferring Files to the HPC
+
+To upload files to DeepThought, the simplest method is to open a Terminal window.
+
+#### The Quick Version
+
+Substitute your filename, FAN and Password, type scp FILENAME FAN@deepthought.flinders.edu.au:/home/FAN then hit enter.
+Enter your password when prompted. This will put the file in your home directory on DeepThought. It looks (when substituted accordingly) similar to:
+
+![SCPExample](../../_static/SCPExampleImage.png)
+
+#### The Longer Version
+
+To download files from DeepThought, you simply need to invert that command to point to either:
+
+- A name of a Computer that Deepthough 'knows' about.
+- An IP Address that Deepthought can reach.
+
+#### Transfers By Computer Name
+
+If you know the hostname of the computer, you can substitute this to transfer files back to your machine. The command stays the same, mostly. You still follow the same idea, we just change where we are pointing it. This one assumed you are transferring it to a Linux/Unix based machine.
+
+The command will take this form:
+
+![SCPByHostname](../../_static/SCPByHostname.png)
+
+#### Transfer By IP Address
+
+If you don't know your computer IP, then the commands of:
+
+- ip addr
+- ifconfig
+
+Will be your friend to figure out what it is. Just like above, we slightly change the command, and sub-in an IP instead of a host-name.
+
+![SCPByIp](../../_static/SCPByIp.png)
+
+### Windows
+
+Windows doesn't support the SFTP protocol in a native way. Thankfully, there are lots of clients written to do just this for us.
+
+#### Sub-System for Linux
+
+You can use the WSL for this - head on over to the [Linux](../Linux/LinuxFileTransfer.md) Guide.
+
+#### Potential Client List
+
+This is not an exhaustive list - feel free to use whatever you wish that supports the SFTP protocol.
+
+- [WinSCP](https://winscp.net/eng/index.php)
+- [FileZilla](https://filezilla-project.org/?AFFILIATE=6732&__c=1)
+
+This guide will focus on WinSCP.
+
+#### Getting Connected
+
+Open WinSCP, enter deepthought.flinders.edu.au as the host to connect to, and click Login. You should have a screen that looks like this.
+
+![WinSCPConnect](../../_static/WinSCPImage.png)
+
+The first time you connect up you will get a warning - this is fine, just click YES to continue on.
+
+![WinSCPSSHKeyNotice](../../_static/WinSCPSSHKeyNotice.png)
+
+A connection to Deep Thought will then be created - login using your FAN and password. If all goes well, you will be treated to this screen:
+
+![WinSCPConnected](../../_static/WinSCPConnected.png)

docs/source/LinuxCommands/LinuxIntro.md

@@ -0,0 +1,11 @@
+# Linux Commands: HPC Style
+
+We try not to re-invent the wheel, and there is already an excellent guide for basic linux commands, written by the excellent people at Harvard. Head on over to [here](https://informatics.fas.harvard.edu/basic-unix-workshop.html) and have a read!
+
+## Tips and Tricks
+
+If you want more from your command line, they have a [useful tips and tricks](https://informatics.fas.harvard.edu/unix-command-line-tips-and-tricks.html) guide that should keep you satisfied.
+
+## Help! I have no idea
+
+Stuck? No idea what Unix is? Linux? BSD? This all complete gibbersh-and-gobbldegook to you? If so, head on over to a [Introduction to UNIX and the Command Line](https://software.rc.fas.harvard.edu/training/intro_unix/latest/#(1)) that should help get you up to speed.

docs/source/ModuleSystem/AdditionalModules.md

@@ -0,0 +1,7 @@
+### EasyBuild Modules
+
+You can opt-in to [EasyBuild](https://easybuild.readthedocs.io/en/latest/#) Managed modules - these are the ones with the strict versioning of all aspects of the install. A brief warning and the list of modules is available [here](EasyBuildModules.md).
+
+### Spack Modules
+
+You can also opt-int to the experimental [Spack](https://spack.io/) modules. These modules are here for testing, but you are free to use them at your own risk! We are running tests to see how we can allow you more power to 'get on with it' and install you own software - Spack is a contender for one of the tools we may use. The list and another warning is over [here](SpackModules.md)