FaceCept3D: 3D Face Analysis and Recognition
Switch branches/tags
Nothing to show
Clone or download
Permalink
Failed to load latest commit information.
FaceCept3D Preparing to push Dec 10, 2015
data Added models for detector and facial expression recognition Dec 10, 2015
docs Minor fixes Aug 14, 2015
papers Added Aug 14, 2015
.gitignore Preparing to push Dec 10, 2015
LICENSE Working on README Aug 13, 2015
README.md Preparing to push Dec 10, 2015

README.md

FaceCept3D

FaceCept3D: 3D Face Analysis and Recognition

Introduction

FaceCept3D is a realtime framework for 3D face analysis and recognition. It contains a set of extendible components that can be combined to fulfil a specific task. At this step we open source the following functionality:

  1. Person-specific template creation
  2. Extreme head pose estimation
  3. Facial expression analysis

FaceCept3D is based on the following works:

  • Robust Real-Time Extreme Head Pose Estimation. [pdf]

  • Facial expression recognition under a wide range of head poses. [pdf]

Here is an example image of head pose estimation using FaceCept3D.

Head Pose Estimation Examples

See FaceCept3D wiki for more details.

Requirements

To install FaceCept3D you need to have the following libraries available

and their dependencies.

In addition, if you plan to use a depth sensor, you will to install the driver. Currently we support only the Microsoft Kinect 1.0 sensor. However, the code can be easily extended to most of the available RGB-D sensors.

To use MS Kinect sensors:

Installation

FaceCept3D was tested on Windows and Linux. Mac users should follow linux installation instructions.

Linux

  • OpenNI and SensorKin drivers. A great guide is here.

  • Prerequsities. Install the following libraries using your package manager:

    boost, eigen, flann, vtk (v 5.6), qhull, opencv, tbb, qt (v 4.8)

    For Ubuntu 14.04 the following command will do the trick:

     sudo apt-get install libboost-all-dev libeigen3-dev libflann-dev libvtk5-dev libqhull-dev libopencv-dev libtbb-dev libqt4-dev

    In addition, you need to install cmake if you don't have it:

     sudo apt-get install cmake
  • Install PCL. We need to build and install pcl, since the version here is built without -std=c++11 modifier. This guide will help you build pcl from source.

  • Check that everything works. Try running pcl_openni_viewier. If you don't see any output, then there is something wrong with your installation.

  • Build FaceCept3D.

    git clone https://github.com/sergeytulyakov/FaceCept3D.git
    cd FaceCept3D
    mkdir build
    cd build
    cmake ..
    make

Windows

On Windows platform there are two ways of build FaceCept3D:

  1. Using CMake to generate an *.sln file
  2. Using the provided *.sln file.

It is somewhat more convenient to use the latter variant since the final solution file is cleaner than the one generated by CMake. If you still want to use CMake, you will need to manually enter all necessary paths.

To simplify the process we provide all the dependencies in a binary format. All libs were compiled using msvc2010-win64. You can still use it in a latter version of Visual Studio, but you need to have msvc2010 toolchain installed.

  • Install Microsoft Kinect Driver 1.8. Download it here and install.

  • Download and unpack precompiled dependencies. Download them here

  • Set up environmental variables You may find useful Rapid Environment Editor for setting the environment on Windows.

    Set KINECTSDK10_DIR and FACECEPT3D_3RDPARTY_DIR. For example, mine are pointing to

     KINECTSDK10_DIR=c:\Program Files\Microsoft SDKs\Kinect\v1.8
     FACECEPT3D_3RDPARTY_DIR=d:\Dependencies\FaceCept3D-3rdparty\

    You also need to add the following folders to your path:

     %FACECEPT3D_3RDPARTY_DIR%\Qt\bin
     %FACECEPT3D_3RDPARTY_DIR%\PCL\bin
     %FACECEPT3D_3RDPARTY_DIR%\Opencv\bin
     %FACECEPT3D_3RDPARTY_DIR%\FLANN\bin
     %FACECEPT3D_3RDPARTY_DIR%\QHull\bin	

    The rest is configured in the Project Properties files. If something doesn't work feel free to inspect the *.props files in the Config folder. If you want to use your versions of the dependencies, you will have to change the *.props files.

    Open 3dheadposeestimation.sln using your version of Visual Studio and try to build. If you have a newer Visual Studio than 2010, do not agree to convert the solution to a newer toolchain, since the dependencies are built with msvc2010-win64 it will not link.

What to do next

Have a look at our wiki pages:

  1. Getting Started
  2. Point Cloud Registration
  3. Personalized Template Creation with FaceCept3D

Citing FaceCept3D

If you use FaceCept3D in your research, please cite one of the following (or both):

@inproceedings{Tulyakov2014,
	author = {Tulyakov, S. and Vieriu, R. L. and Semeniuta, S. and Sebe, N.},
	booktitle = {International Conference on Pattern Recognition},
	title = {{Robust Real-Time Extreme Head Pose Estimation}},
	year = {2014},
}

@inproceedings{Vieriu2015,
	author = {Vieriu, R. L. and Tulyakov, S. and Sangineto, E. and Semeniuta, S. and Sebe, N.},
	booktitle = {Automatic Face and Gesture Recognition},
	title = {{Facial Expression Recognition under a Wide Range of Head Poses}},
	year = {2015},
}