Face Recognition and Face Verification on Nvidia Jetson TX2

Tensorflow implementation of Face Verification and Recognition using th on-board camera of TX2.Facenet and DeepFace implementations for the same are taken as inspiration.These models are compared to a naive K-means clustering approach for recognition tasks.

Demo Link:https://youtu.be/ddQkRRmP6SY

Getting Started

The neural network was trained on Nvidia Titan X GPU.This model was later used with nvidia Jetson TX2 Board. The K-means clustering approach was directly implemented on Jetson TX2.

Prerequisites

1.Python 3.5
2.Tensorflow 1.5
3.Keras
4.Scikit Learn
5.Open CV 3.4.1
4.Face recognition

Installing

I.Download

Pre-trained model weights:

1.FaceNet model weights are directy downloaded from their Github repo.
2.inception_model.py builds the complete network architecture as developed by Szegedy.

II. Open CV

1.Open CV installation for ubuntu is pretty standard and done in same way as shown on their website.
2.Installation of Open CV on Jetson TX2 is a bit diffrent.Jetson Hacks has a pretty cool script on his Git repo.This enable open CV on GPU on the Jetson.

III.Face recognition

sudo pip3 apt-get install face-recognition

This is required for the k-means clustering approach and not the Facenet.This is optional

Running the code

1.Directory Structure

---------------------------------
Face_Recognition
|-fr_utils.py
|-main.py
|-inception_model.py
|-face_recognition_with_jetson.py
|-train-|
|       |-sid1.jpg
|       |-images to train
|-datasets-|
|          |-h5 models datasets as used
|-weights-|
|         |-csv files for tensorflow graph of pretrained Facenet          
------------------------------------

2.Dataset pairs

|-Training will use triplets of images (A,P,N):
|---A is an "Anchor" image--a picture of a person.
|---P is a "Positive" image--a picture of the same person as the Anchor image.
|---N is a "Negative" image--a picture of a different person than the Anchor image.

The facenet aims to minimize this triplet loss.
Here f(A,P,N) stands for the embeddings of each of the input image

3.Run Face verification

verify("train/sid1.jpg", "siddharth", database, FRmodel)
It's Siddharth, hey sexy!
(0.08123432, True)

4.Run Face Recognition

|-Find the encoding from the database that has smallest distance with the target encoding.
|--Initialize the min_dist variable to a large enough number (100). It will help you keep track of what is the closest encoding to the input's encoding.
|--Loop over the database dictionary's names and encodings. To loop use for (name, db_enc) in database.items().
|----Compute L2 distance between the target "encoding" and the current "encoding" from the database.
|----If this distance is less than the min_dist, then set min_dist to dist, and identity to name.

5.Using K-means clustering for Face reogniton

Run the face_recognition_with_jetson.py directly on Jetson.

Results

Both The Facenet and k-means approaches work pretty well.Accuracy wise facenet is much more reliable over large datasets.But k-means is a lightweight option with pretty descent accuracy and does not require huge computation resources.

Authors

• Siddharth Bhonge - Parser /Model - https://github.com/siddharthbhonge

Acknowledgments

• Andrew Ng | Deeplearning.ai
  *Florian Schroff, Dmitry Kalenichenko, James Philbin (2015). FaceNet: A Unified Embedding for Face Recognition and Clustering *Yaniv Taigman, Ming Yang, Marc'Aurelio Ranzato, Lior Wolf (2014). DeepFace: Closing the gap to human-level performance in face verification
  *The pretrained model we use is inspired by Victor Sy Wang's implementation and was loaded using his code: https://github.com/iwantooxxoox/Keras-OpenFace.
  *Our implementation also took a lot of inspiration from the official FaceNet github repository: https://github.com/davidsandberg/facenet