Skip to content

Module 3 FreeSurfer on HiPerGator

Jump to bottom
Jared Tanner edited this page Feb 1, 2025 · 15 revisions

Complete the following

The video linked to below will walk you through setting up and processing the sample data. If you follow along, you should have everything completed for the homework assignment other than uploading the requested files. Remember that subject names, paths, directories, and more might be different. For example, in the video the Module3 directory was in /blue/clp7934/share/Module3 but in 2025 it was in /blue/clp7934/share/clp7934/Module3. Make changes as appropriate.

I did much of the copying and organizing on one of the login nodes. What I recommend is you not do that. It's within the acceptable use of HiperGator to do that, but it's better to do the work in a Console session, a HiPerGator Desktop session, or a SLURM interactive session.

To switch to a SLURM interactive session, you can paste the following for a 2 hour session with 4 GB of RAM. Note that if you close the browser window, it's possible the session will end.

srun --mem=4gb --time=02:00:00 --pty bash -i

Also, this video requests 6 CPU cores. Some students ran out of memory requesting 6 cores and 24GB of RAM. You can request 2-4 cores and keep the amount of RAM (just change the recon-all command to mirror the number of cores: recon-all .... --openmp 4). Or, you can request more RAM. 36GB should be fine but at least one student ran out of RAM with that much, so maybe 48GB. The easiest fix is to cut the number of requested cores down.

NIfTI, BIDS, and FreeSurfer

NOTE: You can load older versions of FreeSurfer. We won't get into why you might but you can do it by specifying a particular version.

The example script you copy (please don't move or delete it!) is in the Module3 directory on the class share. I've also included the contents below.

Process 2 T1 images with FreeSurfer

Use the ADNI_T1_dicom directory in the Module3 directory.

  1. I recommend making a Module3 directory inside yours with a command like this: mkdir /blue/clp7934/share/jjtanner/Module3/
  2. Copy the ADNI_T1w.zip file to your personal directory. The command would look like this: cp /blue/clp7934/share/clp7934/Module3/ADNI_T1w.zip /blue/clp7934/share/jjtanner/Module3/
  3. Convert DICOM files to NIfTI as shown in the video.
  4. Format the T1 files into a simple BIDS format. In the video I end up with directories called sub-011S6303 and sub-011S6367 when making a simple BIDs organization. Make sure the IDs match what data you use. Also, feel free to simplify the ID if you want to. For example, sub-011S6303 could instead be sub-6303.
  5. Process 2 brains using FreeSurfer.

Script to process with notes

Be aware that you will need to update this script to work for you. This is requesting fewer cores (CPUs) than in the walk-through video.

  1. Copy out the script commands and then paste in Visual Studio Code (strongly recommended to use).
  2. Save it with a name like recon-all_sub-6303.sh

Within the script

  1. Put your email address in the appropriate line.
  2. Update the sub_dir variable to match where your data are
  3. Change this line to fit where your data are: cd /blue/clp7934/share/jjtanner
  4. Update the recon-all -s sub-6303 -i command to input the data to where yours are. Also make sure the output subject following the -s flag matches which subject you are processing.

Second script and other notes

  1. Make a copy of the script, calling it with the ID of the other subject. Update all references to the subject in it to match the new one.
  2. If using a Windows computer, when you upload it to HiPerGator, you will likely need to run this in the Terminal: dos2unix recon-all_sub-6303.sh
  3. Assuming both scripts are in your Module3 directory, you could submit the scripts like this sbatch /blue/clp7934/share/jjtanner/Module3/recon-all_sub-6303.sh ; sbatch /blue/clp7934/share/jjtanner/Module3/recon-all_sub-6367.sh

Sample Script

#!/bin/bash
#SBATCH --job-name=recon-all_sub-6303    # Job name
#SBATCH --mail-type=END,FAIL          # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=user@email.domain     # Where to send mail	
#SBATCH --ntasks=1                    # Run on a single CPU
#SBATCH --cpus-per-task=4            # Use 4 cores on node
#SBATCH --mem=24gb                     # Job memory request
#SBATCH --time=12:00:00               # Time limit hrs:min:sec
#SBATCH --account=clp7934            # Allocation name (req'd if you have more than 1)
#SBATCH --qos=clp7934-b                 # QOS selecting burst allocation
#SBATCH --output=recon-all_sub-6303_%j.log   # Standard output and error log
pwd; hostname; date

module load freesurfer/7.4.1

# Define the output subjects directory
sub_dir="/blue/clp7934/share/jjtanner/ADNI_T1w/ADNI_bids/derivatives/freesurfer"

# Check if the directory exists. If not, create it
if [ ! -d "$sub_dir" ]; then
  echo "Directory $sub_dir does not exist. Creating it now."
  mkdir -p "$sub_dir"
else
  echo "Directory $sub_dir already exists."
fi

# Export the variable
export SUBJECTS_DIR=$sub_dir

cd /blue/clp7934/share/jjtanner

recon-all -s sub-6303 -i /blue/clp7934/share/jjtanner/ADNI_T1w/ADNI_bids/sub-6303/ses-01/anat/sub-6303_ses-01_T1w.nii.gz -all -qcache -parallel -openmp 4

date

What you will turn in

  1. screenshot of your BIDS directory.
  2. The recon-all.log files for the brains

Please verify that the jobs completed. The end of the log file should have something like this: recon-all -s sub-6303 finished without error

If you don't see that, the job did not complete appropriately, even if you did not receive an error message. I recommend deleting all files created by the script (the .nii or .nii.gz input files and the output directory for the subject) and trying again. You might need to give the script more RAM (try 36 GB if using 4 or 6 CPUs) or time.

Clone this wiki locally