Emotion recognition from face features

This project is aimed to train model that detects emotion from face image.

Install prerequisites

Using conda

# with conda
conda env update
python get_models.py
# to activate environment
source activate emotion-recogntion-snet-agent

Using pip

# to utilize pip
pip install -r requirements.txt
python get_models.py

How to preprocess datasets

This proejct uses CK+ dataset and Kaggle fer2013 dataset.
The dataset should be saved inside single directory which contains train and test folders.

• To extract kaggle's dataset follow the instruction to download the dataset here
  • Extract the dataset
  • Inside this projects directory run the following code on terminal

python -m preprocess_fer -d /path-to-fer2013.csv-extracted 

• To process the ck+ dataset go to this repository
• After preprocessing both dataset merge the two datasets mannually

How to run training program

The four inputs model can be trained by three steps

• shape_predictor should be inside root directory of this project. Shape predictor can be downloaded to project using the following script.

cd /path-to-project
wget "http://dlib.net/files/shape_predictor_68_face_landmarks.dat.bz2"
bzip2 -d shape_predictor_68_face_landmarks.dat.bz2

# or
python get_models.py

Training program can be run using the following command

python -m train [options]

Option to train program are

• -d : Dataset directory path that contains train and test folders
• -e : Number of epochs to train the model
• -b : Training batch size
• -s : Steps per epoch
• -f : This option specifies the type of model to train. the options can be 'image', 'dlib' or 'all'. The default is 'all'. If user enters option that is not from ('image', 'dlib', 'all') then program continues with default option. Before training the 'all' model type the other two models should be trained and saved inside 'logs/models/' folder.
• -l : learning rate
• -i : image input shape

Step 1 - Training image input model

python -m train [options] -f image 

Step 3 - Training dlib features input model

python -m train [options] -f dlib 

Step 3 - Training the main model

python -m train [options] 

Running Demo program

python -m demo [options]

Options for demo program are

• -j : model json file
• -w : model weights
• -i : Face image source. This could be either image, video or webcam. Defualt is webcam.
• -p : Path to source file. If options -i is webcam this is not necessary.

Requirements

• keras >= 2.0
• tensorflow >= 1.5
• dlib
• opencv >= 3.1
• scikit-image
• numpy
• pandas

Using as singularitynet service

Trained models are available to be run as services to provide emotional results for image currently. It's based on the (alpha-service-example)[https://github.com/singnet/alpha-service-example]. And thus most of the parts are similar.

cp snet.config.example snet.config 
# add private key on snet.config
  `"PRIVATE_KEY": "#PRIVATEKEY GOES HERE"`
python run-snet-service.py --daemon-config-path snet.config

# to test
python test_rpc_call.py

Accessing on Daap

Go to http://alpha.singularitynet.io/ to create a job

TODO

• Better data responses to queries. As it stands we just serialize the data as string. But it's better to utilize grpcs to have consistent message format to communicate with other services or just for single user.
• Persistant hosting to make it work always

LICENSE

MIT License

Copyright (c) 2018 Mitiku Yohannes

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.