CCR Resources

• Intro to CCR course on UBLearns

Prerequisites

• You have to be connected to the UB network to access the CCR. If you're on campus, connecting to Eduroam works. If off campus, connect to the UB VPN via Cisco Secure Client.
• The CCR uses LINUX as the operating system. You need to know and use UNIX commands in the ssh-agent and in the CCR OnDemand portal.
• You'll use Slurm commands to submit job scripts

Note

• Replace {UBIT} with your own ubit name, {GITHUBusername} with your github username, {CCRusername} with your CCR user name (usually the same as your UBIT)
• In Windows, home ~ refers to the folder C:\Users\{windows_username}

Logging in to the CCR

You have two options for logging in

• SSH agent
• OnDemand portal

Both of these options allow you to submit jobs to clusters, manage files etc. Using the OnDemand portal to upload files is easy but can create issues keeping track of old and new versions of files. Once you log in, navigate to the Files then Home Directory at the top of the page to manage files. You can start an interactive session which gives a graphical user interface to interact with the CCR. For more information, see using the OnDemand portal.

Before your first log in using the SSH

Detailed instructions to complete the steps below can be followed from the CCR or Github

1. Generate a SSH key pair (public and private)

ssh-keygen -t ed25519 -C "{UBIT}@buffalo.edu"

• You'll be prompted to create a SSH passphrase. Create something memorable. You'll need to use this passphrase each time to get connceted to the CCR.

2. Add the private SSH key to the SSH agent

eval "$(ssh-agent -s)"

3. Add the public SSH key to the CCR Identity Management Portal This may take up to 30 minutes to allow you to log in

• Open the public key located at ~/.ssh/id_ed25519.pub and copy the contents from there OR
• In the ssh agent use cat to view the contents of the file and copy the contents from there

cat ~/.ssh/id_ed25519.pub

• Login to the CCR IDM portal
• Click on SSH Keys in the left navigation menu
• Click on the "New SSH Key" button, paste the contents of your public key in the text box, and click "Add"

Logging in regularly

ssh {CCRusername}@vortex.ccr.buffalo.edu

Transferring Files to the CCR

There's lots of ways. Choose whatever works best for you. Information can be found here I recommend using Github because it manages file versions and handles data conflicts well.

Using Github for file management

Generate SSH key

ssh-keygen -t ed25519 -C "{UBIT}@buffalo.edu"

Press enter to save ssh key to default file. Then enter a passphrase (please create a passphrase!)

Add SSH key to the ssh-agent

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

Should say identity added and the location e.g. Identity added: /user/{UBIT}/.ssh/id_ed25519 ({UBIT}@buffalo.edu)

Add SSH public-key to your Github

1. Use cat to view the contents of the file and copy the contents from there

cat ~/.ssh/id_ed25519.pub

2. Copy and paste that whole line to Github > Settings > SSH and GPG keys

Test SSH Connection

ssh -T git@github.com

Ignore the warning. Type yes.

Clone Repository into your folder on CCR

git clone --branch {branch} git@github.com:{GITHUBusername}/repositoryname

Maintaining file management

Recommended: make all changes on your local PC only to avoid merge conflicts

git pull origin {branch}

Other commands such as git push and git stash will come in handy. You can find useful git commands here

Running code with a slurm script

Prequisite

• You should already be logged in to the CCR

Simplest Example using "Hello World"

1. Create a python file to print "Hello World". Vim is just one of the Linux default text editors.

vim hello.py

2. Press i to set mode to insert and write to the file
3. File contents should look something like this

#!/usr/bin/env python3
print("Hello World")

4. Press esc to exit insert mode. Type :wq to save the file and quit the text editor
5. Back on command line, create slurm script and open it in a text editor.

vim run.sh

6. Write to the file to submit a job to a debug node on the CCR.

#!/bin/bash
#SBATCH --partition=debug
#SBATCH --qos=debug
python  ./hello.py 

7. Back on the command line, submit the job

sbatch run.sh

8. View the output file Tip: Pressing Tab autocompletes file names within the directory

cat {slurm-jobid.out}

Simple examle using Gurobi

1. Download the file knapsack.py to your local computer
2. Use scp in the SSH agent to copy the file from your computer to the CCR (must be connected to UB network for this to work) Tip: If your folder name contains whitespaces wrap the folder name in single quotation marks. For example, 'CCR Workshop'/knapsack.py

scp -v {local-path-to-file} {CCRusername}@vortex.ccr.buffalo.edu.edu:/user/{CCRusername}/{CCR-path-to-file}/.

3. Log into the CCR

ssh {CCRusername}@vortex.ccr.buffalo.edu

4. View the knapsack.py file in the text editor to check what modules we need to load

vim knapsack.py

5. Check the module dependecies

module spider gurobi

4. Create a slurm script to run the gurobi file

vim test.sh

5. Add slurm commands to the slurm script.

#!/bin/bash

#SBATCH --cluster=ub-hpc
#SBATCH --partition=general-compute
#SBATCH --qos=general-compute
#SBATCH --mail-user={UBIT}@buffalo.edu
#SBATCH --mail-type=end
#SBATCH --time=00:03:00
#SBATCH --job-name="knapsack"
#SBATCH --output=knapsack.out
#SBATCH --error=knapsack.err
#SBATCH --ntasks=1
#SBATCH --mem=64M

6. Add module loads to the slurm script

module load gcccore/11.2.0
module load gurobi/10.0.1

7. Add instructions to run the python file to the slurm script

python ./knapsack.py

8. Back on command line, run the slurm script

sbatch test.sh

9. View output of the run

cat knapsack.out

Alternatively, if the run was not succesful error codes would be written to

cat knapsack.err

Simple example using job array

1. Download the files job.sh and jobarray.py to your local computer
2. Copy the files from your computer to the same directory on the CCR
3. Log into the CCR
4. Navigate to directory you saved the files in. Make the slurm script executable

chmod +x job.sh

5. Convert script with DOS line breaks to Unix line breaks

dos2unix job.sh

6. Run job

sbatch job.sh

7. Check progress of the job

squeue --user={UBIT}

8. Check progress of the job every x seconds

squeue --user={UBIT} --iterate=x

Stop with CTRL-C 9. For help with slurm commands

sbatch --help

To be added:

• Why use the CCR
• Creating a python virtual environment to load modules not already on the CCR
• Multithreading vs Multiprocessing in Python