Skip to content

Module 3 FreeSurfer on HiPerGator

Jump to bottom
Jared Tanner edited this page Dec 15, 2025 · 15 revisions

Module 3: FreeSurfer on HiPerGator – Comprehensive Tutorial

Overview

This guide covers the process of converting raw MRI data (DICOM) into NIfTI format, organizing it into a simplified BIDS structure, and processing it using FreeSurfer on the HiPerGator supercomputer.

Prerequisites:

  • Access to HiPerGator.
  • Basic familiarity with the command line (Terminal).
  • Visual Studio Code (recommended for editing scripts).

Part 1: Setting Up Your Environment

Crucial Rule: Do not run heavy processing tasks on the "Login Node" (the screen you see immediately after logging in). You must request a compute node.

1. Start an Interactive Session

Instead of queuing a job immediately, we will do our file setup in an "Interactive Session." This gives you a dedicated computer node to work on.

  1. Open your terminal/console on HiPerGator.
  2. Paste the following command to request a 2-hour session with 4GB of RAM: 
    srun --mem=4gb --time=02:00:00 --pty bash -i
    Wait for the prompt to change (e.g., from user@login to user@c0123).

2. Define Your Directories

To make this tutorial work for you, you need to know where your folder is.

  • Source Data: /blue/psy4930/share/psy4930/Module3 (This is where the class data lives).
  • Your Directory: Likely /blue/psy4930/share/YOUR_USERNAME.

3. Create Your Module Directory

Run the following commands to create your workspace and copy the raw data. Replace jjtanner with your actual username.

1. Create a Module3 directory in your share folder

mkdir -p /blue/psy4930/share/jjtanner/Module3/

2. Enter that directory

cd /blue/psy4930/share/jjtanner/Module3/

3. Copy the raw DICOM zip file from the class share

cp /blue/psy4930/share/psy4930/Module3/ADNI_T1w.zip .

4. Unzip the data

unzip ADNI_T1w.zip

Part 2: DICOM to NIfTI Conversion

MRI scanners produce DICOM files. Neuroimaging software (like FreeSurfer) prefers NIfTI files. We typically use a tool called dcm2niix for this.

1. Load the converter module

On HiPerGator, dcm2niix is bundled with MRIcroGL. Run this specific command to load the tool and its dependencies:

module load mricrogl gcc/5.2.0 pigz

2. Convert the images

We will convert the DICOMs and output them into a new folder called nifti_output. (Note: The command below assumes the unzipped ADNI_T1w folder contains the DICOM subfolders).

Create a temporary folder for NIfTIs

mkdir nifti_output

Run conversion

Notes about the flags for options.

-z y: Compresses the output (creates .nii.gz)
-o: Output directory
-f: Filename format (%p=protocol, %i=ID)

Command to run. dcm2niix -z y -o nifti_output -f "%p_%i" ADNI_T1w

Check your nifti_output folder (ls nifti_output). You should see .nii.gz files corresponding to the subjects.

Part 3: BIDS Formatting

BIDS (Brain Imaging Data Structure) is the standard for organizing data. We will create a simplified version: Subject -> Session -> Type -> File.

Goal Structure:

Module3/
└── ADNI_bids/
    ├── sub-6303/
    │   └── ses-01/
    │       └── anat/
    │           └── sub-6303_ses-01_T1w.nii.gz
    └── sub-6367/
        └── ses-01/
            └── anat/
                └── sub-6367_ses-01_T1w.nii.gz

Steps to Organize:

Run these commands to move your converted files into the correct structure.

# 1. Create the BIDS root directory
mkdir ADNI_bids

# --- Setup Subject 1 (6303) ---
# Create the folder tree
mkdir -p ADNI_bids/sub-6303/ses-01/anat

# Move and Rename the file (Replace 'ORIGINAL_FILENAME_6303.nii.gz' with the actual name inside nifti_output)
mv nifti_output/*6303*.nii.gz ADNI_bids/sub-6303/ses-01/anat/sub-6303_ses-01_T1w.nii.gz

# --- Setup Subject 2 (6367) ---
# Create the folder tree
mkdir -p ADNI_bids/sub-6367/ses-01/anat

# Move and Rename the file
mv nifti_output/*6367*.nii.gz ADNI_bids/sub-6367/ses-01/anat/sub-6367_ses-01_T1w.nii.gz

Note: Depending on how dcm2niix named your files, you may need to use ls nifti_output to see the exact names to use in the mv command.

Part 4: FreeSurfer Processing Script

Now we will write the script to process the data. This script submits a "Job" to the cluster so you don't have to keep your computer on while it runs.

1. Create the Script

Open VS Code (or a text editor). Copy the code below. You MUST edit the lines marked with <--- CHANGE THIS.

#!/bin/bash
#SBATCH --job-name=recon-all_sub-6303    # Job name
#SBATCH --mail-type=END,FAIL             # Mail events (NONE, BEGIN, END, FAIL, ALL)
#SBATCH --mail-user=YOUR_EMAIL@ufl.edu   # <--- CHANGE THIS: Where to send mail
#SBATCH --ntasks=1                       # Run on a single CPU task
#SBATCH --cpus-per-task=4                # Use 4 cores on node
#SBATCH --mem=24gb                       # Job memory request (Try 36gb if this fails)
#SBATCH --time=12:00:00                  # Time limit hrs:min:sec
#SBATCH --account=psy4930                # Allocation name
#SBATCH --qos=psy4930-b                  # Burst allocation
#SBATCH --output=recon-all_sub-6303_%j.log   # Standard output log
pwd; hostname; date

# Load FreeSurfer
module load freesurfer/7.4.1

# --- DEFINING PATHS ---

# 1. Set your base directory (Where you did your work in Part 1)
# <--- CHANGE THIS to match your path (e.g., /blue/psy4930/share/your_name/Module3)
export BASE_DIR="/blue/psy4930/share/jjtanner/Module3" 

# 2. Define where FreeSurfer should save results (Derivatives)
export SUBJECTS_DIR="$BASE_DIR/ADNI_bids/derivatives/freesurfer"

# Check if output directory exists. If not, create it
if [ ! -d "$SUBJECTS_DIR" ]; then
  echo "Directory $SUBJECTS_DIR does not exist. Creating it now."
  mkdir -p "$SUBJECTS_DIR"
else
  echo "Directory $SUBJECTS_DIR already exists."
fi

# 3. Define the Input Image (NIfTI file)
# Ensure this matches the path you created in Part 3
INPUT_IMG="$BASE_DIR/ADNI_bids/sub-6303/ses-01/anat/sub-6303_ses-01_T1w.nii.gz"

# --- RUNNING FREESURFER ---

echo "Starting Recon-all for sub-6303..."

# -s: Subject ID (name of the output folder)
# -i: Input NIfTI file
# -all: Run the full pipeline
# -qcache: Pre-calculate stats
# -parallel -openmp 4: Use 4 CPUs (matches --cpus-per-task above)

recon-all -s sub-6303 \
          -i "$INPUT_IMG" \
          -all \
          -qcache \
          -parallel \
          -openmp 4

echo "Finished."
date

2. Save the Script

Save this file as recon-all_sub-6303.sh.

3. Create Script for Subject 2

Save a copy of the script as recon-all_sub-6367.sh

Find and replace every instance of 6303 with 6367 in the new file.

4. Upload and Fix Format (Crucial for Windows Users)

Upload your .sh files to your Module3 directory on HiPerGator. If you edited these files on Windows, you must run a special command to fix hidden formatting characters that break Linux scripts:

Run inside your Module3 directory on HiPerGator

dos2unix recon-all_sub-6303.sh
dos2unix recon-all_sub-6367.sh

Part 5: Submit and Verify

1. Submit the Jobs

Run these commands in the terminal:

sbatch recon-all_sub-6303.sh
sbatch recon-all_sub-6367.sh

The system will give you a Job ID (e.g., Submitted batch job 1234567).

2. Monitor

You can check if they are running by typing:

squeue -u YOUR_USERNAME

3. Verify Success

FreeSurfer takes several hours (6-12 hours usually). Once the squeue command shows nothing, the job is done.

Check the Logs: Look for a file named recon-all_sub-6303_XXXXXXX.log in your directory. Open it (using cat or less) and scroll to the very bottom. You must see this line: recon-all -s sub-6303 finished without error

Check the Output Data: Check your BIDS derivatives folder:

ls ADNI_bids/derivatives/freesurfer/sub-6303/scripts/ Ensure the data is populated.

Troubleshooting / Common Errors

"Out of Memory" / Job Killed:

If the log stops abruptly without the "finished without error" message, you likely ran out of RAM.

Edit your script: Change #SBATCH --mem=24gb to #SBATCH --mem=36gb.

Delete the partial subject folder in derivatives/freesurfer/sub-XXXX before restarting.

Command not found:

Did you run dos2unix on your script?

Did you ensure module load freesurfer/7.4.1 is in the script?

Path errors:

Double-check that you updated the BASE_DIR in the script to point to your folder, not jjtanner.

Deliverables for Homework

A screenshot of your ADNI_bids directory structure (showing the sub/ses/anat hierarchy).

The .log files generated by SLURM (the text files proving the job finished without error).

Clone this wiki locally