├── data
│ ├── bookid
│ ├── face
│ ├── meta
│ └── test
├── face2bmi_mt.py
├── face2bmi.py
├── full.csv
├── img
│ ├── avengers_comparison.jpg
│ ├── detect_predict_multi_faces.png
│ ├── model_structure.jpg
│ ├── mtcnn_face_alignment.jpg
│ ├── tensorboard_results.png
│ └── trump_comparison.jpg
├── models.py
├── multi_task_end_to_end.ipynb
├── notebook.ipynb
├── prediction.ipynb
├── README.md
├── requirements.txt
├── saved_model
│ └── models_vgg16_fc6.pkl
├── tasks.py
├── test.csv
├── train.csv
├── train_vggface_model_feature_extraction.ipynb
└── web_scrape.py
This is FUTURE 2.0! Now even a single picture of your face can predict your BMI without knowing your height & weight. Not even a single face, multiple faces can be deteced in a single pisture to predict the AGE, SEX & BMI of all the visible faces on that picture. Isn't that futuristic?
The model provides end-to-end capability of detecting faces and predicting the BMI, Age and Gender for each person in the same picture.
The architecture of the model is described as below:
Face detection is done by MTCNN, which is able to detect multiple faces within an image and draw the bounding box for each faces.
It serves two purposes for this project:
Prior model training, each image is preprocessed by MTCNN to extract faces and crop images to focus on the facial part. The cropped images are saved and used to train the model in later part.
Illustration of face alignment:
In inference phase, faces will be detected from the input image. For each face, it will go through the same preprocssing and make the predictions.
Illustration of ability to predict for multiple faces:
In vanilla CNN architecture, convolution blocks are followed by the dense layers to make output the prediction. In a naive implementation, we can build 3 models to predict BMI, age and gender individually. However, there is a strong drawback that 3 models are required to be trained and serialized separately, which drastically increases the maintenance efforts.
[input image] => [VGG16] => [dense layers] => [BMI] |
[input image] => [VGG16] => [dense layers] => [AGE] |
[input image] => [VGG16] => [dense layers] => [SEX] |
Since we are going to predict BMI, Age, Sex from the same image, we can share the same backbone for the three different prediction heads and hence only one model will be maintained.
[input image] => [VGG16] => [separate dense layers] x3 => weighted([BMI], [AGE], [SEX]) |
This is the most simplified multi-task learning structure, which assumed independent tasks and hence separate dense layers were used for each head. Other research such as Deep Relationship Networks, used matrix priors to model the relationship between tasks.
In this project Tech invloved as follows:
MT-CNN, Cv2, VggFace, Keras, Tensorflow
For better understanding :
Here you can access the web-application.
Screen.Recording.2022-09-05.at.3.53.33.PM.mov
Learned how to use MtCNN for reducing the processing time and increasing the efficiency to get a better optimised algorithm to process multiple CNN models simultaneously and get a unified output.
VggFace library is pretty handy to process, allign, and agument human faces.
Due to lack of enough data set of asian specially Indian faces, the model is not as accurate when applied on Indian faces as expected. Providing a good ammount of data set, this model can predict bmi as efficiently as it does in case of western faces.
I am a AI and Machine Learning Enthusiast & growing Android Developer (kotlin). Both the fields, Machine Learning and Android Development, fascinates me a lot. And I also have worked on Azure Cloud Computing platform to deploy machine learning models.
