This project focuses on developing deep learning models based on convolutional neural network to perform the automated semantic image segmentation of the MR images of brain. We explore the current state of the art autoencoder style U-Net architecture, also we use other prominent CNNs such as ResNet and VGG as a backbone in the U-Net architecture and evaluate them on the BraTS dataset. Different regularisation methods and hyperparameters are tested and optimised through a series of experiments. Finally, a web application is created so that the developed models can be used easily by medical practitioners.
>= 2.0.3
- Numpy >= 1.21.5
- keras >= 2.6.0
- matplotlib >= 3.5.2
Install the requirements using command below:
pip install -r requirements.txt- nibabel
- SimpleITK
- pyradiomics
- Tensorflow
- Keras
- segmentationmodels3d
To setup a local copy to run the project follow the instructions:
Available here
All BraTS multimodal scans are available as NIfTI files (.nii.gz) and describe a) native (T1) and b) post-contrast T1-weighted (T1Gd), c) T2-weighted (T2), and d) T2 Fluid Attenuated Inversion Recovery (T2-FLAIR) volumes. All the imaging datasets have been segmented manually, by one to four raters, following the same annotation protocol, and their annotations were approved by experienced neuro-radiologists. Annotations comprise the GD-enhancing tumor (ET — label 4), the peritumoral edema (ED — label 2), and the necrotic and non-enhancing tumor core (NCR/NET — label 1)
Run the Preprocessing.ipynb to preprocess the .nii MRI files to yield numpy arrays.
Run Master.ipynb on the preprocessed dataset to yield 3D U-Net model. Run Master VGG + ResNet.ipynb to get deep learning models based on U-Net architecture with VGG or ResNet as backbone.
Run Web app/app.py to start the localhost application. Upload an untested preprocessed MRI image to generate the segmentation. Images generated for each slice of MRI scan are stored in Web app/Uploads folder. Swap the brats_3d.hdf5 model with the newly generated model to use it.
The results of the experiments and a more detailed description of the theoretical background, the used resources/methods and the general usage of this repository can be found in the report.
Result for proposed 3D U-Net model on testing data :
Sample prediction :
