A Siamese neural network (sometimes called a twin neural network) is an artificial neural network that uses the same weights while working in tandem on two different input vectors to compute comparable output vectors [From WIKI]. In the modern Deep learning era, Neural networks are almost good at every task, but these neural networks rely on more data to perform well. But, for certain problems like face recognition and signature verification, we can’t always rely on getting more data, to solve this kind of task we have a new type of neural network architecture called Siamese Networks. It uses only a few numbers of images to get better predictions. The ability to learn from very little data made Siamese networks more popular in recent years [From Towards Data Science]. This repository contains my solution to human face verification with One-shot learning using siamese neural network. The dataset used for this problem is the Celebrities in Frontal-Profile in the Wild (CFPW dataset).
This project requires Python and the following Python packages:
If you do not have Python installed yet, it is highly recommended that you install the Anaconda distribution of Python, which already has most of the above packages.
CFPW dataset contains images of celebrities in frontal and profile views. There are 500 celebrities' images in 500 folders. All of the folders have 10 frontal view images and 4 profile view images, so a total of 14 images are in each and every folder. For training the siamese neural network I randomly choose 350 folders and for the validation set, I randomly choose 100 folders and, the rest of the 50 folders were kept for the test set. The test set remained isolated the whole time, I just it at last just to verify if my model has accuracy in the out-of-sample examples.
- siamese_augmentation.py- As the input in the siamese network needs to be given as duplets of positive examples (same class) or negative examples (different class), I was unable to use the Keras image augmentation method. So, I wrote the siamese_augmentation.py file. This file and its function are responsible for the augmentation of the input duplets in the training data. Example of a positive duplet image.
-
siamese_function.py- This file is responsible for creating duplets from the data. First of all image folders are randomly sampled from the data. Then the sampled folder images are loaded in the NumPy array. Images are loaded in RGB format. From the image data, it will generate batches of positive duplets and negative duplets continuously. This file is also responsible for the visualization of duplets, if necessary.
-
siamese_model.py- The siamese network I used have almost 13M trainable parameter. The CNN architecture used in this repository is similar to the VGG16 but the model I used has less trainable params(13M) compared to the original VGG16(138.4M) model and the model I used also have BatchNormalization layers for regularization purposes. Written in this medium article about siamese network and visualizing its general architecture.
"Siamese Network is a One-shot classifier with two mirror-image subnetworks to rank similarity or dissimilarity between the two inputs using Similarity score."
- siamese_testdata.py- From the isolated test set, randomly 1500 duplets of positive examples and negative examples are selected. Those 1500 duplets are used to verify siamese model accuracy in the out-of-sample examples. The accuracy score from the test set is 83.267%. The confusion matrix for the test data set.
- siamese_enroll.py- This file is responsible for testing new enrollment data. "siamese_enroll_dataset" folder contains the enrolled images. "siamese_input_image" folder contains the captured test image. "siamese_input_image" folder image is compared with "siamese_enroll_dataset" folder images. If a similarity between images is found, then it is reported. I used my own image to test the network. Note that my data is not used in training, validation, or test set.
The siamese neural network, I trained did not converge fully. Though the test set accuracy is 83.267%, the lowest validation loss is 0.32531. I also used learning rate decay to control the oscillator.
Training loss Vs Validation loss graph
Training accuracy Vs Validation accuracy graph
Learning rate decay graph
