BASH tutorial (Linux and MacOS command prompt)
By Etienne Aumont
- Basic BASH commands
1.1 BASH cheatsheet
1.1 Navigating through files
1.2 Manipulating files - For loops
2.1 Rationale
2.2 Basic loop structure + examples
2.3 Additional tools to name outputs
2.4 Multi-step for loops
2.5 Loops with 2 variables - Advanced functions
3.1 Grabbing information
3.2 Running things in the background
3.3 Editing files directly in the terminal
Commands
cd Current Directory - to move around
pwd Prints what the current directory is
ls Lists files and directories
tree Show directory and file tree
mkdir Creates a directory
cp Copies a file or a directory
mv Moves a file (like cut and paste)
rm Deletes a file
rmdir Deletes a directory
echo Displays what comes after
tar For un-compression. (Use option -xzf for files ending in .tar.gz)
historyTo access your terminal history
grep A search function that can be used to print a specific piece of information
nohup To run programs in the background
cat To grab string (text) information from a file
ssh To connect to a server or different machine, either through the local network or on the Internet
Action keys & tips
- "Tab" key: Autocomplete (for files/directories)
- Tab (twice): Displays options for autocomplete
- CTRL + C: Interrupts current command (you can only copy + paste using the right mouse click)
- Middle mouse click: Copies whatever is selected in the command prompt (faster than copy + paste)
- -help: Type
--helpafter any command to display the command's options
In the command prompt, you are by default working from the home directory on your drive. Every command you use will be from the point of view of this directory.
First, to change your point of view, you can move through the directories with cd. For example:
cd /home/minc/Desktop/FEOBV/analyses/
This will bring me from the home directory to my FEOBV analyses directory.
You can always press TAB twice to autocomplete or display your options while typing the path
Files with spaces in their name will not work here, make sure you remove all spaces!
cd alone will bring you back to the home directory
To go back, use .., for example:
cd ../../MK6240/analyses/
This will bring me 2 directories out from the current directory - the Desktop - then into a different analyses folder.
If you ever wonder where you currently are, you can use pwd to display the path to the current directory
Next, you may list files in the current directory using ls
This will display everything, so the list may be long. If it's too long, you can always press CTRL + C to interrupt the current command.
It's also possible to display the content of another file with ls path/to/your/folder/
The content will by default be organized by name, but you can organize them by modification time by adding "-t" right after "ls".
To see the file structure, you can use tree instead
You can create new directories using mkdir. For example:
pwd
>/home/minc/Desktop/MK6240/analyses
mkdir hippocampus
ls
>hippocampus
You can copy this directory around using cp and including the option -r for directories. Then populate it with an image.Instead of copying a file, you can also move it using mv. This is also useful to rename your file
cp -r hippocampus ../../FEOBV/analyses/
cd ../../FEOBV/analyses/
cp ../../FEOBV001_SUVR.mnc hippocampus/FEOBV001_SUVR.mnc
mv hippocampus/FEOBV001_SUVR.mnc hippocampus/NEW_NAME001_SUVR.mnc
Instead of copying a file, you can also move it using mv
This is also useful to rename your file
To delete stuff, use rm for files or rmdir for directories.
Be careful with this action, as it is permanent
To limit mistakes, you can use echo first to make sure that you've got it right.
echo rm hippocampus/NEW_NAME001_SUVR.mnc
>rm hippocampus/NEW_NAME001_SUVR.mnc
rm hippocampus/NEW_NAME001_SUVR.mnc
rmdir hippocampus
The folder hippocampus no longer exists
Other uses for echo:
echo content > patient_parameters.txt #a txt file containing the patient's weight and injected dose
> 89.5
> 6.5
For loops allow you to perform operations on all objects that correspond to your criteria. This is a very efficient way of, for example, processing neuroimaging data.
For loops require you to first define the objects you're interested in (eg. all MK6240 images).
In BASH, this usually means using something in the file's name to identify it.
Using MINC functions, you can name an output based on the input. However, this process is based on the length of the file names, so this only works well if the files are named consistently. Ideally, all IDs should be the exact same length to allow you to name the outputs efficiently.
Also, I recommend all of your files either be in the same folder or be nested into a subject folder.
Your folder could therefore look like this:
PersonalFolder/MK/SUVR/
for [VarName] in [criteria for the specific files you want] ;
do [The command you want to be executed on all selected files] ;
done
You refer to your file in the command using $VarName When creating a new file based on parts of the existing file name, you can use the file name using
${VarName:0:18} #using 18 characters starting from character 0
Example: Display a series of images on Display (they will open 1 at a time, and the next one will automatically open when you close the previous one)
for file in * ;
do Display -gray -range 50 900 "${file:0:10}"*.mnc -label ../Template_labels_Left/"${file:0:10}"*.mnc ;
done
Example: obtain the mean SUVR within a mask File structure is [ID(7 character long)]/[ID+filetype]
for file in * ;
do mincstats -mask ../WM_0.99_new.mnc -mask_binvalue 1 "${file:0:7}"/"${file:0:7}"*avg_suv_autoreg_ST.mnc -mean ;
done
The output will be a series of SUVR values based on the file's alphabetical order
FOR loops are also super useful to collect and move files around. For example:
for i in */*T2*.mnc ; #Collecting all T2 images in minc format nested 2 layers deeper than the current directory
do cp $i ../../../etienne/ ; #Copying all the images inside my folder
done
Adding one character to a file name to make it longer:
!!! If you modify a file like this, I recommend you run it with the echo function first to see if it's well built.
for i in MRT*.mnc ; #Collecting all files with "MRT" in the ID
do echo mv $i "${i:0:7}"O"${i:7:100}" ; #Adding an "O" at the end of the ID to bring it to 8 characters long
done
#output (1 line per file):
mv MRT6123_segmented_labels.mnc MRT6123O_segmented_labels.mnc
Also, remember that if something goes wrong, you can stop the for loop using the CTRL + C keys
The function looks good, we can now run it:
for i in MRT*.mnc ; #Collecting all files with "MRT" in the ID
do mv $i "${i:0:7}"O"${i:7:100}" ; #Adding an "O" at the end of the ID to bring it to 8 characters long
done
Inputting text file values into the loop
for I in $(cat path/to/file.txt) ; do echo $i ; done
#will print all of the values in the .txt file. It reads the lines one by one.
Character number work fine when names have the same lengths at all times. Other simple tools can be used for when it's not the case such as referring to landmarks within the name such as "." and "/".
%.* removes anything after the first ".", very useful for mni2nii and nii2mnc functions!
export i=x/y/file.nii.gz #to create our variable
echo ${i%.*}
> x/y/file
echo ${i%%.*}
> x/y/file.nii #it removes what comes after the 2nd "."
It's the same idea for #*., which removes everything before the first instance of what is right after the "."
echo ${i#*.}
> nii.gz
echo ${i##*/}
> file.nii.gz
There is also basename --, which uses a different syntax to remove the directory path to keep only the file name. This is useful when you want to add a prefix to a processed file.
echo $(basename -- "$i")
> file.nii.gz
You can add steps in your for loop and use the output of the first step in the second one. To do so, use something like
REF=$($echo [command you want the output of])
The output must ONLY be what you're interested in. In Minc commands, this generally implies using -quiet Echo means outputting the command's result as text
Example: Standardize a PET image based on the signal in a ROI
for file in * ;
do REF=$($echo mincstats -mask ../../refMask_fullCG_res.mnc -mask_binvalue 1 $file -mean -quiet) ;
mincmath -div -const $REF "${file:0:15}"*_ST.mnc "${file:0:18}"_SUVR_ST.mnc ; done
A second example of ref region SUVR
for i in FEOBV*.mnc ; do REF=$($echo mincstats -mask c2xSVBSS_er.mnc -mask_binvalue 1 $i -mean -quiet) ;
mincmath -div -const $REF $i s"${i:0:13}"_PrevST_SUVR.mnc ; done
Adding equations into the mix
Here, I want to automatically calculate the optimal range for Display based on the white matter signal intensity
for i in train00* ;
do REF=$($echo mincstats -mask $i/seg_left_t.mnc -mask_binvalue 20 $i/n4_mprage.mnc -mean -quiet);
MIN=$(echo "$REF*0.275" |bc -l) ; #calculating the min with an equation based on the WM value
MAX=$(echo "$REF*1.1" |bc -l);
Display -gray -range $MIN $MAX $i/n4_mprage.mnc -label $i/seg_left.mnc ;
done
The for function doesn't support more than one variable. However, you can use while to create 2 or more variables from a list file. The list file should look like this:
701_9301_20221104|701_9301_20221108
701_9302_20221104|701_9302_20221118
701_9303_20221104|701_9303_20221202
701_9304_20221104|701_9304_20221128
701_9305_20221104|701_9305_20230106
701_9306_20230309|701_9306_20221115
701_9307_20221219|701_9307_20221125
701_9308_20230105|701_9308_20221122
701_9308_20240514|701_9308_20240530
In this case, it contains IDs for 2 imaging modalities that were performed on a different date. I used this to create merged folders containing both image modalities like this:
cat /home/etienne/Desktop/list2.txt | while IFS='|' read i j ;
do cp $j/${j:0:14}_t1.mnc ../MK/$i/${j:0:8}_t1.mnc ;
done
The grep function allows us to specify the part of an output we want to see. For example, I had a sheet with all PET files. I needed to obtain the information about the frames from all of them. With mincheader, I determined that the line I was interested in always began with ' time = '
for I in $(cat MT6240.txt) ; do mincheader $i | grep ' time = '
As a result, we have a list of the lines containing the information.
However, in some cases, one line isn't enough. We can add information such as context for the dynamic PET images (-C), those that start with 6 seconds frames
for I in $(cat MT6240.txt) ; do mincheader $i | grep -C 3 ' time = 0, 6,'
The result isn't great as a list, but at least it gives all the information.
Using the nohup function, you can run tasks in the background. They will keep running even if you close the terminal or if you log out.
The function also allows you to store a log file so that you can later investigate if something went wrong.
This is useful when running long, computer-intensive tasks. The format looks like this:
nohup bash -c 'your command' > output.log &
For example, I run partial volume correction on a reference region mask here and use the output to calculate the SUVR:
nohup bash -c 'for i in N*/*_ADNI.mnc ;
do REF=$($echo petpvc -i $i -m ../refMask_infCG.mnc -o ${i:0:14}/ref.txt --pvc gtm -x 2.4 -y 2.4 -z 2.4 |
egrep -v 'Performing Geometric matrix method...') ; #to remove the petPVC processing message
mincmath -div -const $REF ${i:0:14}/${i:0:14}_native.mnc ${i:0:14}/${i:0:14}_native_SUVR.mnc ;
done' > outputBroca.log &
The nano function allows you to open code files and edit them. It's not the best text editor, but it can be convenient.
For example, you can edit your configuration file using
nano .bashrc
This will allow you, for example, to set python (or conda) to activate when you open a new terminal.
To move around, you can scroll or use the arrows. use the Ctrl key to use the options identified at the bottom (i.e. press Ctrl + x to exit). Here, I've added:
source /data/data05/vfonov/conda/etc/profile.d/conda.sh
It was commented out and replaced automatically by conda.
I also wanted conda to automatically launch my own environment, which I had previously created. To do so, I added:
conda activate etienne
My environment is set within my own disc space, so I could install new packages.