This projects served as part of a requirement for the Foundations of Deep Learning at CentraleSupélec. The task is a semantic segmentation one requiring not only the prediction of of 26 different possible classes within an image but to generate masks to indicate where the classes exist within the images. Since the dataset is of images from Hurricane Harvey, the main objective would be to accurately determine segments of images that have been affected by flooding and identify the objects in the vicintiy of the flood. The project was a challenging one and a successful deployment could yield to an assistance in quicker indentification of flood zones to dispatch rescue teams.
Note: Only sample images have been added to this repository for illustration. Actual training was done on full dataset
The project is done by Karim El Hage, Ruining Ma
1 - Understanding of the Class Distribution within the images
2 - Image Augmentation and Patching: Training Images are augmented using several techniques and split into segment patches
3 - Modeling, Evaluation, and Comparison: The model is trained using PSPNet and evalauted using a weighted multi-F1 dice score and Accuracy in Class Prediction
4- Demonstration of Algorithm: A few demonstration of the model's predictive capabilities compared to the True masks
A demonstration of the results can be found below. Each image (left) in the provided training set had an associated mask generated by humans (middle). The prediction of the model's mask can then also be visualized (right). The best model trained was a model with a PSP-Net architecture acheiving a F1 dice score of 74% on an unseen testing set. We can see that the model is able to correctly predict classers in general but fails to work with the finer details in an mask.
It has been observed during training that several training masks do not reflect the exact representation of an image, especially during the instance of a flooding. In the below example, we can see how a validation mask has not trees shown in a flooded region whereas the model predicted the exposed trees in the flooded area. This leads to the model outputting a lower accuracy even though it has technically produced the correct class. In future work, the dataset can be explored in detail to create a better image augmentation plan to integrate minority classes better and to deal with the situation of incorrect true masks.
A special shoutout to Dr. Sreenivas Bhattiprolu who's learning material on Image Augmentation and Semantic Segmentation have been a true help to understanding the application of Semantic Segmentation. Link to his repo: https://github.com/bnsreenu/python_for_microscopists