BASH tutorial (Linux and MacOS command prompt)
By Etienne Aumont
- Basic BASH commands
1.1 BASH cheatsheet
1.1 Navigating through files
1.2 Manipulating files - For loops
2.1 Rationale
2.2 Basic loop structure + examples
2.3 Multi-step for loops
2.4 Loops with 2 variables - Advanced functions
3.1 Grabbing information
3.2 Running things in the background
3.3 Editing files directly in the terminal
3.4 Running things on another computer
Commands
cd Current Directory - to move around
pwd Prints what the current directory is
ls Lists files and directories
tree Show directory and file tree
mkdir Creates a directory
cp Copies a file or a directory
mv Moves a file (like cut and paste)
rm Deletes a file
rmdir Deletes a directory
echo Displays what comes after
tar For un-compression. (Use option -xzf for files ending in .tar.gz)
historyTo access your terminal history
grep A search function that can be used to print a specific piece of information
nohup To run programs in the background
cat To grab string (text) information from a file
ssh To connect to a server or different machine
Action keys & tips
- "Tab" key: Autocomplete (for files/directories)
- Tab (twice): Displays options for autocomplete
- CTRL + C: Interrupts current command (you can only copy + paste using the right mouse click)
- Middle mouse click: Copies whatever is selected in the command prompt (faster than copy + paste)
- -help: Type
--helpafter any command to display the command's options
In the command prompt, you are by default working from the home directory on your drive. Every command you use will be from the point of view of this directory.
First, to change your point of view, you can move through the directories with cd. For example:
cd /home/minc/Desktop/FEOBV/analyses/
This will bring me from the home directory to my FEOBV analyses directory.
You can always press TAB twice to autocomplete or display your options while typing the path
Files with spaces in their name will not work here, make sure you remove all spaces!
cd alone will bring you back to the home directory
To go back, use .., for example:
cd ../../MK6240/analyses/
This will bring me 2 directories out from the current directory - the Desktop - then into a different analyses folder.
If you ever wonder where you currently are, you can use pwd to display the path to the current directory
Next, you may list files in the current directory using ls
This will display everything, so the list may be long. If it's too long, you can always press CTRL + C to interrupt the current command.
It's also possible to display the content of another file with ls path/to/your/folder/
The content will by default be organized by name, but you can organize them by modification time by adding "-t" right after "ls".
To see the file structure, you can use tree instead
You can create new directories using mkdir. For example:
pwd
>/home/minc/Desktop/MK6240/analyses
mkdir hippocampus
ls
>hippocampus
You can copy this directory around using cp and including the option -r for directories. Then populate it with an image.Instead of copying a file, you can also move it using mv. This is also useful to rename your file
cp -r hippocampus ../../FEOBV/analyses/
cd ../../FEOBV/analyses/
cp ../../FEOBV001_SUVR.mnc hippocampus/FEOBV001_SUVR.mnc
mv hippocampus/FEOBV001_SUVR.mnc hippocampus/NEW_NAME001_SUVR.mnc
Instead of copying a file, you can also move it using mv
This is also useful to rename your file
To delete stuff, use rm for files or rmdir for directories.
Be careful with this action, as it is permanent
To limit mistakes, you can use echo first to make sure that you've got it right.
echo rm hippocampus/NEW_NAME001_SUVR.mnc
>rm hippocampus/NEW_NAME001_SUVR.mnc
rm hippocampus/NEW_NAME001_SUVR.mnc
rmdir hippocampus
The folder hippocampus no longer exists
Other uses for echo:
echo content > patient_parameters.txt #a txt file containing the patient's weight and injected dose
> 89.5
> 6.5
For loops allow you to perform operations on all objects that correspond to your criteria. This is a very efficient way of, for example, processing neuroimaging data.
For loops require you to first define the objects you're interested in (eg. all MK6240 images).
In BASH, this usually means using something in the file's name to identify it.
Using MINC functions, you can name an output based on the input. However, this process is based on the length of the file names, so this only works well if the files are named consistently. Ideally, all IDs should be the exact same length to allow you to name the outputs efficiently.
Also, I recommend all of your files either be in the same folder or be nested into a subject folder.
Your folder could therefore look like this:
PersonalFolder/MK/SUVR/
for [VarName] in [criteria for the specific files you want] ;
do [The command you want to be executed on all selected files] ;
done
You refer to your file in the command using $VarName When creating a new file based on parts of the existing file name, you can use the file name using
${VarName:0:18} #using 18 characters starting from character 0
Example: Display a series of images on Display (they will open 1 at a time, and the next one will automatically open when you close the previous one)
for file in * ; do Display -gray -range 50 900 "${file:0:10}"*.mnc -label ../Template_labels_Left/"${file:0:10}"*.mnc ; done
Example: obtain the mean SUVR within a mask File structure is [ID(7 character long)]/[ID+filetype]
for file in * ;
do mincstats -mask ../WM_0.99_new.mnc -mask_binvalue 1 "${file:0:7}"/"${file:0:7}"-FEOBV-*avg_suv_autoreg_ST.mnc -mean ;
done
The output will be a series of SUVR values based on the file's alphabetical order
FOR loops are also super useful to collect and move files around. For example:
for i in */*T2*.mnc ; #Collecting all T2 images in minc format nested 2 layers deeper than the current directory
do cp $i ../../../etienne/ ; #Copying all the images inside my folder
done
Adding one character to a file name to make it longer:
!!! If you modify a file like this, I recommend you run it with the echo function first to see if it's well built.
for i in MRT*.mnc ; #Collecting all files with "MRT" in the ID
do echo mv $i "${i:0:7}"O"${i:7:100}" ; #Adding an "O" at the end of the ID to bring it to 8 characters long
done
#output (1 line per file):
mv MRT6123_segmented_labels.mnc MRT6123O_segmented_labels.mnc
Also, remember that if something goes wrong, you can stop the for loop using the CTRL + C keys
The function looks good, we can now run it:
for i in MRT*.mnc ; #Collecting all files with "MRT" in the ID
do mv $i "${i:0:7}"O"${i:7:100}" ; #Adding an "O" at the end of the ID to bring it to 8 characters long
done
Inputting text file values into the loop
for I in $(cat path/to/file.txt) ; do echo $i ; done
#will print all of the values in the .txt file. It reads the lines one by one.
You can add steps in your for loop and use the output of the first step in the second one. To do so, use something like
REF=$($echo [command you want the output of])
The output must ONLY be what you're interested in. In Minc commands, this generally implies using -quiet Echo means outputting the command's result as text
Example: Standardize a PET image based on the signal in a ROI
for file in * ; do REF=$($echo mincstats -mask ../../refMask_fullCG_res.mnc -mask_binvalue 1 $file -mean -quiet) ;
mincmath -div -const $REF "${file:0:15}"*_ST.mnc "${file:0:18}"_SUVR_ST.mnc ; done
A second example of ref region SUVR
for i in FEOBV*.mnc ; do REF=$($echo mincstats -mask c2xSVBSS_er.mnc -mask_binvalue 1 $i -mean -quiet) ;
mincmath -div -const $REF $i s"${i:0:13}"_PrevST_SUVR.mnc ; done
The for function doesn't support more than one variable. However, you can use while to create 2 or more variables from a list file. The list file should look like this:
701_9301_20221104|701_9301_20221108
701_9302_20221104|701_9302_20221118
701_9303_20221104|701_9303_20221202
701_9304_20221104|701_9304_20221128
701_9305_20221104|701_9305_20230106
701_9306_20230309|701_9306_20221115
701_9307_20221219|701_9307_20221125
701_9308_20230105|701_9308_20221122
701_9308_20240514|701_9308_20240530
In this case, it contains IDs for 2 imaging modalities that were performed on a different date. I used this to create merged folders containing both image modalities like this:
cat /home/etienne/Desktop/list2.txt | while IFS='|' read i j ;
do cp $j/${j:0:14}_t1.mnc ../MK/$i/${j:0:8}_t1.mnc ;
done
Using the nohup function, you can run tasks in the background. They will keep running even if you close the terminal or if you log out.
The function also allows you to store a log file so that you can later investigate if something went wrong.
This is useful when running long, computer-intensive tasks. The format looks like this:
nohup bash -c 'your command' > output.log &
For example, I run partial volume correction on a reference region mask here and use the output to calculate the SUVR:
nohup bash -c 'for i in N*/*_ADNI.mnc ;
do REF=$($echo petpvc -i $i -m ../refMask_infCG.mnc -o ${i:0:14}/ref.txt --pvc gtm -x 2.4 -y 2.4 -z 2.4 |
egrep -v 'Performing Geometric matrix method...') ; #to remove the petPVC processing message
mincmath -div -const $REF ${i:0:14}/${i:0:14}_native.mnc ${i:0:14}/${i:0:14}_native_SUVR.mnc ;
done' > outputBroca.log &
The nano function allows you to open code files and edit them. It's not the best text editor, but it can be convenient.
For example, you can edit your configuration file using
nano .bashrc
This will allow you, for example, to set python (or conda) to activate when you open a new terminal.
To move around, you can scroll or use the arrows. use the Ctrl key to use the options identified at the bottom (i.e. press Ctrl + x to exit). Here, I've added:
source /data/data05/vfonov/conda/etc/profile.d/conda.sh
It was commented out and replaced automatically by conda.
I also wanted conda to automatically launch my own environment, which I had previously created. To do so, I added:
conda activate etienne
My environment is set within my own disc space, so I could install new packages.
ssh command
ssh command is used to switch to a different machine on Linux systems. It is useful for our TNL lab operations because most of the better machines are located outside the lab. If option '-Y' is specified, it would allow graphics forwarding, so that programs with graphical user interface (like RStudio, MATLAB, etc) will look the same way as on the local machine.
Example:
ssh -Y anton
After that, start your RStudio, MATLAB, etc from that terminal, the program will then run on the specified machine - in this case, the machine called anton.
List of fast machines that can be used for heavier computations:
adie, agraval, anton, aran, bastian, beck, beevor, bender