Skip to content
Snapchat-alike filters app using opencv
Branch: master
Clone or download
Latest commit bc31da8 Aug 23, 2017
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
filters last changes Aug 10, 2017
imgs update Aug 14, 2017
sprites last changes Aug 10, 2017
LICENSE
README.md Update README.md Aug 23, 2017
facial_features.py
facial_landmarks.py
main.py bugs fix Aug 11, 2017
main_dlib.py bugs fix Aug 11, 2017
requirements.txt update Jul 23, 2017

README.md

Snapchat-like filters with Python OpenCV/dlib and TKinter

Basic desktop application to play around with Snapchat-alike filters like hat, moustache and glasses automatic in-face superposition in real time.

alt text alt text

It uses Haar features and the Viola–Jones object detection framework implemented in OpenCV to detect mainly faces positions and inside the faces, eyes and mouth position. It uses then this information to add different accessories to the faces (hat, moustache, etc).

The files containing the Haar Filters descriptions were taken from http://alereimondo.no-ip.org/OpenCV/34 and ftp://mozart.dis.ulpgc.es/pub/Software/HaarClassifiers/FaceFeaturesDetectors.zip.

The Dlib implementation of this app uses a Histogram of Oriented Gradients (HOG) feature combined with a linear classifier, an image pyramid, and sliding window detection scheme to detect faces. Then it finds the 68 facial landmarks using an Ensemble of Regression Trees described in the paper One Millisecond Face Alignment with an Ensemble of Regression Trees.

Requirements

  • OpenCV 3.0+ with python bindings
  • Python 2.7
    • pillow
    • numpy
    • tkinter
  • Python bindings of dlib.

Easy install

Build OpenCV or install the light version with sudo apt-get install libopencv-dev python-opencv. For Windows users it is always easier to just download the binaries of OpenCV and execute them, see this web page. For TKinter in Linux just execute: apt-get install python-tk (python binaries in windows usually have Tkinter already installed).

For python dependences just execute:

pip install -r requirements.txt

Dlib installation: For Windows Users it can be very hard to compile dlib with python bindings, just follow this simple instruction on how to install dlib in windows the easy way.

For Linux Users make sure you have the prerequisites:

sudo apt-get install build-essential cmake
sudo apt-get install libgtk-3-dev
sudo apt-get install libboost-all-dev

Finally just pip install it with: pip install dlib

How it works

HaarClassifiers

The algorithm give us the rectangles where faces might be as it can be seen in the next picture. After that, specialized HaarClassifiers are used to find mouths, noses, and eyes inside the image recognized as a face. With that information it is relatively easy to put the different "gadgets" in superposition with the image. We just use the alpha channel (transparency) of the "accessories" to merge the images.

alt text

To recreate this in real time with your WebCam just execute

python facial_features.py

Dlib face and landmarks detection

HarrCascades technique is very old, so detection is often not so good when the faces are slightly bent for example. It is not possible to estimate the inclination of the head for example. Detection of face characteristics (eyes, nose, mouth, eyebrows, etc) is also not robust with HaarClassifiers. Here comes into play more recent techniques implemented in the dlib library. If you compare the to detection schemes the dlib is more accurate and stable. As described, we can also get the main facial landmarks which are used to detect the face characteristics and to estimate the tilt angle of the face. That can be seen in the next picture.

alt text

The 68 facial landmarks are numbered in the following form. With tha points we calculate the inclination so the "accessories" are also bent. Wth the points of the mouth can be detected when the mouth is open and then display some trick, like a rainbow coming out.

alt text

To recreate this in real time with your WebCam just execute

python facial_landmarks.py

Running the code with opencv or dlib

In Windows just double click de file main.py or execute in the console the following to run the HaarCascade version app (opencv)

python main.py

To use the dlib version with more interesting features run

python main_dlib.py

Comparison between OpenCv HaarCascade Classifiers and Dlib algorithms

Here can be compared the performance in real time of the two techniques. As seen the opencv with HaarCascade implementation is not so "stable" (i.e the hat moves a lot although the face is still). Also with the dlib version more fun things can be made and the accessories inclines with us.

OpenCV

alt text

Dlib

alt text

You can’t perform that action at this time.