Skip to content
Mutli-modal research project that combines text analysis and image processing to determine emotions
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.

Emotion Detection

Research project that focuses on recognizing emotions using face recognition and NLP. The project is developed by Emory NLP lab.


This repository contains source files from two repositories: face-py-faster-rcnn, which subsequently uses py-faster-rcnn


  1. Software requirements
  2. Installation of Faster R-CNN
  3. Adjusting to detect in Friends TV show
  4. Face_recognition package installation
  5. Run Face_recognition package

Software requirements

  1. Install caffe and pycaffe. Detailed installation instructions can be found at Caffe: Installation
  2. Python Packages: cython, opencv-mac, or opencv-ubuntu, and easydict
  3. Make sure you use virtual environment. Further instructions can be found at pyimagesearch

Installation of Faster R-CNN (Based on face-py-faster-rcnn)

  1. Follow every step outlined by face-py-faster-rcnn

Note 1: Make sure Step 3 and Step 4 are executed without errors, otherwise, caffe won't run.

Note 2: Face-py-faster-rcnn uses WIDER data set to train the model. The output should be .caffemodel extension

Note 3: Make sure to download the package Faster RCNN package 'recursively'.

  1. To test the model, the tutorial uses FDDB provided by UMass. So, if you are using pre-trained model, there is no need to download WIDER data set (make sure you adjust code accordingly).
  2. If all the steps are executed without errors, you can run the code on dataset:
  ~$ cd rcnn/face-py-faster-rcnn-master/ 
  python ./tools/ --gpu=0

Installation of Faster R-CNN (Friends TV show)

  1. Update directory names in the file:
1.1 location of .caffemodel
1.2 data_dir, out_dir
1.3 plt.savefig() in the vis_detections() method
  1. Run the code as in Step 3 of Installation of face-py-faster-rcnn

**General Note: **

After updating the directory names, if you get 'asType' error, it means that directory isn't being read properly. In other words, recheck by asserting or printing directories you visit.

Installing Face Recognition package using dlib

  1. Install dlib using either Method 1 or Method 2.

Method 1

Use procedure outlined by PyImageSearch

Method 2

Make sure you install python, opencv, boost, boost-python, dlib using brew install package_name.

Download dlib from website

Activate virtual environment, and in dlib directory, run python install

More information can be found in this github

  1. Install face_recognition package

Make sure dlib is installed without errors, otherwise face_recognition will not run properly. Then, in your bash run

pip install face_recognition
  1. More information about face recognition can be found here

  2. Convenient explanation and sample explanations are given by Adam Geitgey

Run Face Recognition Package

  1. Copy file
  2. Inside the file provide 3 paths.
2.1 rootDir - known faces (i.e. faces of characters: Rachel, Ross,  Monica, etc)
2.2 unknownDir - location of pictures where you need to recognize faces 
2.3 newDir - location where to store new, evaluated pictures
You can’t perform that action at this time.