-
Notifications
You must be signed in to change notification settings - Fork 0
2. Accessing the HPC environment
- "How do I log on to the HPC resources?"
- "Learn to access the HPC environment from your OS"
- "Remote shells, EI’s HPC hostname"
The High Performance Computing (HPC) system offers versatile remote access options, whether you're on-site or off-site, utilizing a secure VPN connection. Upon gaining remote access, users are allocated a 'session' or 'shell' on a 'login' node, also known as 'submission' or 'head' nodes. Once connected, users can directly submit commands to HPC nodes or queue larger jobs through the integrated job scheduling system, SLURM.
To initiate a session on the HPC, the process varies depending on your operating system. Assuming that you’re on the campus network:
Accessing NBI HPC (Mac):
-
Open a terminal (cmd+SPACE, then type 'terminal').
-
Enter the command:
ssh USERID@slurm.nbi.ac.ukor simplyssh hpcto log in to the main submission node.
Accessing NBI HPC (Windows):
-
Windows does not have a native terminal system (unless you have installed the Linux subsystem for windows) so instead we’ll be using Putty, a free terminal emulator, serial console, and network file transfer application. Putty is widely used on Windows platforms for connecting to remote servers, routers, and other network devices to perform command-line tasks and configuration.
-
Open the Putty application
-
Enter the command:
ssh USERID@slurm.nbi.ac.ukorssh hpcto log in to the main submission node.
Accessing NBI HPC (Linux):
-
Open a terminal (Ubuntu shortcut: Ctrl+Alt+t).
-
Use the command:
ssh USERID@slurm.nbi.ac.ukorssh hpcto log in to the main submission node.
Additionally, access can be facilitated via VPN (covered separately) or through a browser service:
- Browser access: Visit https://ood.hpc.nbi.ac.uk/ to utilize services such as running GUI applications (e.g., RStudio) and easily uploading/downloading small files to HPC storage.
After establishing a connection, users can perform various actions. For instance:
- Verify your current directory location using the command
pwd; you will be in your home directory (/hpc-home/ also accessible as '~/'), a directory that only you have read or write access to with a capacity of approximately 300GB.
- Induction
- HPC Best practice
- Job Arrays - RC documentation
- Methods to Improve I/O Performance - RC documentation
- Customising your bash profile for ease and efficiency
- Customise bash profile: Logging Your Command History Automatically
- Using the ei-gpu partition on the Earlham Institute computing cluster
- Using the GPUs at EI
- HPC Job Summary Tool
- EI Cloud (CyVerse)
- Git and GitHub
- Worked examples
- Job Arrays
- Using Parabricks on the GPUs
- dependencies
- Software installations
- Workflow management system
- Transfers
- Local (mounting HPC storage)
- Remote - <1gb (ood)
- Remote - <50gb (nbi drop off)
- Remote - No limit (globus)
- mv command
- The nbi-download partition (retrieving external datasets)