This repository contains the code of my diploma thesis.
I use the DeepLabV3+ neural network to extract deep features and I use a Gaussian Mixture Model to perform image segmentation.
I used the DeepLabv3+ neural network with the xception architecture and pretrained weights on the PASCAL VOC 2012 semantic segmentation dataset. The original repo can be found here: https://github.com/tensorflow/models/tree/master/research/deeplab To use this neural network with the Keras framework I used the implementation from this GitHub repo: https://github.com/bonlime/keras-deeplab-v3-plus
I extracted the features from the last convolutional layer of the neural network for a set of color images that I resized to 512x512.
To avoid the curse of dimensionality I reduced the extracted features using Principal Component Analysis (PCA). I used the scikit learn implementation of PCA with n=8
principal components and derived with 8 feature images for each input image.
To perform image segmentation I used a Gaussian Mixture Model with the principal component features. I used the Gaussian Mixture Model implementation from scikit learn for
different numbers of clusters. After fitting the Gaussian Mixture Model with a few images I used it to perform segmentation on new images.
To improve the result of the segmentation I used max, min, and meadian filters as well as morphological tranformations such as opening on the segmented iamges.
To test the results I applied the process on a video, using Gaussian Mixture Models with different numbers of clusters.
To validate the results I compare the results of my system with DeepLab's segmentation.
In addition, I calculated the Intersection over Union (IoU) of the results using annotated images as ground truth. The annotated images were provided by user https://github.com/sfikas.