In this repository you can find a trained model for detecting smiles and open mouth on images. This model uses dlib face and facial landmarks detector.
Train data consists of 865 images from Helen dataset and other sources. Also there additional 40 images from Helen dataset for manual testing.
Faces and landmarks were extracted from source train images using dlib and hand-labeled for smile and open mouth using interactive widgets in Jupyter Notebook. This is performed in preprocess_and_label.ipynb notebook.
A simple Feed-Forward model is trained on 2278 pairwise distances between facial landmarks points. The distances are precomputed for the train set in distances.npy file. Model layers sizes are:
2278 -> 1000 -> 700 -> 500 -> 500 -> 4.
The full architecture is shown on the figure.
After 100 epochs of training using Adam optimizer with learning rate 0.0001 an accuracy of ~ 0.85 is achieved on test data. Manual testing on extra images shows reasonable results on new images.