Face Detection
- Module ID: Detection
- Module Type: Class
- Module purpose: This is the first step in our mobile application. It receives an image as an input and it can be used to detect either the central person of that image or all the faces from the input. Then, the detected faces are saved into different images and a series of processing techniques are applied. The faces are transformed into greyscale and resized so that they have the same attributes as the images from the database used for face recognition. In the next step, to assure a consistent contrast and brightness, a histogram equalization is performed. The resulting face images are sent as an input to the face recognition module.
- Subordinate modules: None
- Dependent modules: Recognition
- Interface: see IDetection.h
Haar Classifier is the face detection method implemented by OpenCV. It is version of Viola-Jones detector, a face detection technique developed by Paul Viola and Michael Jones, based on Haar features, also known as Haar-like wavelets.
A haar feature or more correctly a haar-like wavelet is a combination of adjacent black and white rectangles, which can be used to describe face components ( eyes, mouth, eyebrows). Examples of haar-like wavelets and how can they be used for face detection:
The presence of a haar-like wavelet is determined by subtracting the average dark-region pixel value from the average light-region pixel value. If this difference is greater than a given threshold, we decide that this feature is present.
If an image area contains a series of haar features, then that area will be considered a face, otherwise an element from the background.
More information about the Viola-Jones algorithm can be found here.
The face detection module that will be further used in the application has been developed using OpenCV’s functions for haar classifier. The module has been written using C++ classes.
- Initialize
- cascadeOption – it is used to specify the classifier cascade used for face detection. Different classifiers should be used if we want to detect frontal or profile faces. The default classifier will detect the frontal faces.
- scaleOption – this parameter is used to specify the rate at which the haar features used for face detection will be scaled. A lower scale option means that more faces will be detected, while a higher scale option will perform a faster detection, but may miss some faces from the input image. The default scale value is 1.1, that determines an increase in the features dimension of 10% at each step.
- minHitsOption – this parameter tells us how many times an image area should be detected in order to consider it a face. If we set this option to 0 or 1 a face will be detected multiple times, while setting it to a higher value ( like 5 ) will lead in missing a large number of faces. The default value is 2, which detects almost all the faces from a random input image. The faces that are not double detected, will be discarded in a later step.
- templateSizeOption – it is used to specify the minimal area in which to search for a face. If we want to detect persons from close-up images, the size should be over 40 pixels, otherwise a 25 region pixels (which is the default value ) is enough for detecting a large number of faces.
- resultImageWidth – the width of the detected faces images. This parameter (along with the resultImageHeight ) is used to specify the dimension of the images that contain the detected persons and which will be passed as an argument to the face recognition module. The default value is 92.
-
resultImageHeight – the height of the detected faces images. The default
height value used is 112. - detectObjects
In order to create a flexible face detection implementation, that can be parametrized according to the desired results, this module can be initialized with the following arguments:
All this parameters can be set using the initialize method. The syntax for setting a parameter is “-optionName=value”. If the initialize method is not called, the default values will be used.
This is the most important method of the module, as it accomplishes the main task, the detection of the faces. This function receives as a first argument an input image in the OpenCV’s image format IplImage*. Given an image file’s path, the IplImage* can be loaded like this:
IplImage* image = cvLoadImage(imagePath, CV_LOAD_IMAGE_COLOR);
The nrOfDetectedFaces parameter can be used in order to specify how many faces we want to detect in an image. The value of the parameter can be set using the two predefined constants: DETECT_ALL_FACES and DETECT_A_FACE. Before applying face detection, the input image should be preprocessed. The image is converted into greyscale and the contrast and brightness are equalized:
// convert image to grayscale
detectImage = cvCreateImage( cvSize(image→width,image→height), 8, 1 );
cvCvtColor( image, detectImage, CV_BGR2GRAY );
// equalize brightness and contrast
cvEqualizeHist( detectImage, detectImage );
The next step is performing the actual detection:
//detect faces from image
detectedObjects = cvHaarDetectObjects(
detectImage, // input image
this→cascade_, // haar classifier
storage, // memory for detected faces
this→scaleValue_, // scale factor for cascade classifier
this→minNeighbours_, // nr of overlapping detection for an object to be considered face
this→flags_, // detection settings
minTemplateSize // smallest region in which to search for a face);
The flags that can be set are:
CV_HAAR_DO_CANNY_PRUNING – skip regions with no lines
CV_HAAR_SCALE_IMAGE – scale the image, and not the feature
( more time consuming, but can offer better results)
CV_HAAR_FIND_BIGGEST_OBJECT – return only the biggest object
CV_HAAR_DO_ROUGH_SEARCH – stop the search when the first candidate is found
The result, detectedObjects represent a vector of rectangles that correspond to the areas of the input image where a face was detected.
This method receives as arguments an IplImage* and a vector of rectangles that represents the detected faces in that image. As a result, the method returns a vector of IplImage* that are images of the detected faces, processed and resized to resultImageWidth*resultImageHeight. The method’s result is used by the face recognition module to identify the detected faces.
Here is a short example of how to use the face detection module in order to detect the faces from an image and save the detected faces into separate images.
//initialize with default parameters
//for other settings, you should call the ‘initialize’ function
Detection* detection = new Detection();
// load image
IplImage* image = cvLoadImage(imageName, CV_LOAD_IMAGE_COLOR);
// detect faces from an IplImage and return the coordinate of each face
vector objects = detection→detectObjects(image,DETECT_ALL_FACES);
// save all detected faces from an IplImage in a vector of IplImage ( each vector component
// is a IplImage that contains a detected face )
vector<IplImage*> faces = detection→getDetectedObjects(image, objects);
// save detected faces from a vector of IplImage into a directory
detection→saveImage(faces,dirName);
delete detection;
A full example of using the face detection module for locating all the faces from a series of images can be found at TestDetection.cpp .
To use this file in order to test the detection module perform the following commands:
make
./detection fileName
The fileName parameter is a file in which you can list the images you want to recognize people from ( An example can be found here).
The results will be located in the result folder ( if it does not exist, it will be created automatically).
Face detection performance can be analyzed if we monitor three different things:
- detection rate – nr of faces detected by the algorithm/nr of faces detected by a human
- false positives – nr of faces detected by the algorithm that are not actually a face
- false negatives – nr of faces undetected by the algorithm
Obviously, it is desirable to have a detection rate close to 100% and the number of false positives and negatives should be close to 0.
In order to have conclusive results, it is essential to use a great variety of test images. There should be included both color and grayscale images and both images with one face and with multiple persons.
My three datasets used for testing are:
- Database 1 – contains color images of multiple persons. The lighting conditions vary in each image, the persons can be closer or further away in the image and the background is cluttered.
- Database 2 – contains single face images in different lighting conditions and backgrounds. The facial expressions of people’s faces vary.
- Database 3 – contains grayscale images of single or multiple faces oriented in different positions.
Here are the face detection results on the 3 databases:
| Image No. | Nr of subjects | Detection rate | False positives | False negatives |
|---|---|---|---|---|
| 1.jpg | 12 | 75% | 0 | 3 |
| 7.jpg | 33 | 64% | 0 | 12 |
| 14.jpg | 7 | 100% | 0 | 0 |
| 18.jpg | 17 | 100% | 1 | 0 |
| 32.jpg | 4 | 100% | 0 | 0 |
| 38.jpg | 14 | 100% | 0 | 0 |
| 41.jpg | 10 | 100% | 0 | 0 |
| 74.jpg | 10 | 90% | 0 | 1 |
| 96.jpg | 17 | 94% | 0 | 1 |
| 131.jpg | 2 | 100% | 0 | 0 |
The overall detection rate is 92.3%.
This second database contains images with only a face per image. No false positives were encountered and the detection rate was 96.6%.
This database is very popular for testing frontal face detection, combining images from CMU and MIT.
| Image Name | Nr of subjects | Detection rate | False positives | False negatives |
|---|---|---|---|---|
| aerosmith.jpg | 5 | 100% | 0 | 0 |
| albert.jpg | 1 | 100% | 0 | 0 |
| audrey2.jpg | 1 | 100% | 0 | 0 |
| ew-courtney-david.jpg | 2 | 100% | 0 | 0 |
| ew-friends.jpg | 6 | 83% | 0 | 1 |
| hendrix2.jpg | 1 | 100% | 0 | 0 |
| judybats.jpg | 5 | 100% | 0 | 0 |
| madaboutyou.jpg | 2 | 50% | 0 | 1 |
| married.jpg | 3 | 100% | 0 | 0 |
| mickymouse-self-p.jpg | 2 | 100% | 0 | 0 |
| monalisa.jpg | 1 | 100% | 0 | 0 |
| seinfeld.jpg | 4 | 100% | 0 | 0 |
As you can see on the table, the results when using this database were pretty good ( 94.41% average detection rate ). Only two persons have not been detected and no false positives have been encountered. The fact that the images were originally grayscale was not an important factor, since the detection module converts the input image to grayscale before applying the face identification algorithm.
