Iris partitions

Partition	Walltime	Purpose
interactive	<= 24 hours	For interactive jobs only
cpushort	<= 2 hours	CPU short jobs
gpushort	<= 2 hours	GPU short jobs
cpu (default)	<= 7 days	CPU regular jobs
cpu_highmem	<= 7 days	Nodes with 4TB of RAM
gpu	<= 7 days	GPU regular jobs
chanj3	<= 7 days	PI buy-in resources
preemptable*	<= 7 days	Jobs can be preempted if resources needed in above partitions

*Intended for low-priority jobs that Slurm can stop to allow high-priority jobs to run; because of this, jobs submitted to the preemptable partition need to have checkpoints in place. For example, let's say I've designed a job with checkpoints, and I want it to run whenever a particular server type is available, but I'm okay with if Slurm needs to stop my job to let a high-priority job run. If a high-priority job is submitted by someone else (say, they have priority in their own partition for that server), since my job is preemptable, Slurm will stop my job and start the other user's. Once the resources become available again, my job will restart from the last checkpoint.

See more details here: https://github.mskcc.org/pages/HPC/userdocs/scheduler/slurm/partitions.html

chanj3 partition limit

scontrol show partition chanj3

PartitionName=chanj3
   AllowGroups=ALL AllowAccounts=hpcadmins,chanj3 AllowQos=ALL
   AllocNodes=ALL Default=NO QoS=N/A
   DefaultTime=02:00:00 DisableRootJobs=NO ExclusiveUser=NO ExclusiveTopo=NO GraceTime=0 Hidden=NO
   MaxNodes=UNLIMITED MaxTime=7-00:00:00 MinNodes=0 LLN=NO MaxCPUsPerNode=UNLIMITED MaxCPUsPerSocket=UNLIMITED
   Nodes=iscf018,isch002
   PriorityJobFactor=1 PriorityTier=2 RootOnly=NO ReqResv=NO OverSubscribe=NO
   OverTimeLimit=NONE PreemptMode=OFF
   State=UP TotalCPUs=128 TotalNodes=2 SelectTypeParameters=NONE
   JobDefaults=DefCpuPerGPU=1
   DefMemPerCPU=2048 MaxMemPerNode=UNLIMITED
   TRES=cpu=120,mem=2062533M,node=2,billing=120,gres/gpu=8,gres/gpu:h100=4,gres/gpu:l40s=4

Request for GPUs from chanj3 partition

Interactively:

srun -p chanj3 -w iscf018 --gpus=1 --pty bash # L40S GPU
srun -p chanj3 -w isch002 --gpus=1 --pty bash # H100 GPU

Group user limit

sacctmgr show assoc account=chanj3 format=User,Partition,Account,QOS,GrpTRES,MaxTRES,MaxJobs

Fairshare priority

• sprio -u $USER: job-level
• sshare -u $USER: user-level

Check resource usage after jobs finished

seff: efficiency of the resources

Start time estimate

squeue --start -j <jobid>: If it prints NA, the scheduler can't yet estimate.

Storage

Shared data

• SAIL: /data1/collab002/sail/shared/data/collaborators/rudin-charles-lab/
• MSK-IMPACT BAM files: /data1/share001/share/impact_12_245/
• MSK-IMPACT key file: /data1/share001/share/impact_12_245.key.txt
• MSK-ACCESS BAM files: /data1/share001/share/access_12_245/
• MSK-ACCESS key file: /data1/share001/share/access_12_245.key.txt