Wahl-O-Selfie - v2

Description

Wahl-O-Selfie v2 is using facial recognition to categorize human faces into (german) political partys similar to "Wahl-O-Mat¹".

Table of Contents

• Description
• Table of Contents
• How to install
  • Installing face-recognition (for Windows)
  • Installling the other dependency
  • Installing Wahl-O-Selfie
• How to use
• How it works
• Problems
• License and credits
• Postscript

How to install

Wahl-O-Selfie (v2) requires:

• Python 3.12.0
• face_recognition
  • Microsoft Visual Studio 2015 (or newer)
  • CMake
  • dlib
• pillow
• os
• pickle

Installing face_recognition (for Windows)²

To install face_recognition you have to have Python 3.12.0 and Microsoft Visual Studio 2015 (or newer) with C/C++ Compiler installed. After that you have to install CMake for Windows and add it to your system variables. Then you have to install dlib using pip:

pip install dlib

and finally you can install the face_recognition library, also by using pip:

pip install face_recognition

Installing the other dependency

Finally, you can install the other dependency with pip:

pip install Pillow

The libraries os, pickle and time are pre-installed with Python

Installing Wahl-O-Selfie

Download the project as a .zip file and unzip iton your machine. Then open main.py with a code editor, preferably with Microsoft Visual Stusio Code, but make sure that you have the Python extension installed.

How to use

After running the program a new file, called result.jpg by default, apears in the program folder. In this picture you can see the face of a man surrounded by a rectangle with the predicted party below it, as seen here:

If you want to try the program on one of your pictures you have to copy it into the program directory and either rename it to testimage.jpg (if your picture is a jpg file) or you have to change the first parameter in the last line in the main.py file:

predictParty('testimage.jpg', 'result.jpg', False)

The first parameter ('testimage.jpg') is the picture that will be loaded to be analyzed. The second parameter ('result.jpg') is the name of the picture that will be saved after the analyzation. The third parameter (which is currently set to False) determines if the program should print (cool looking) lines inbetween the face landmarks which are used to identify the faces. These lines look like this:

cool, eh?

In the program directory I also included a second testimage you can try the program on. (Both of the testimages were created with This Person does not exist, just in case you wonder.)

You can also create your own dataset to be used to identify people. More instructions can be found in "face_data/raw/readme.txt".

How it works

After running the 'predictParty' function the program procedes to run the 'loadKnown' function in 'prepare_faces.py'. The 'loadKnown' function then checks if the three required files 'face_data/encodings.dat', 'face_data/names.dat' and 'face_data/partys.dat' exist. If they exist the program loads the variables saved in these files, which contain the required informations needed by the program to identify and associated the correct names and partys to the faces it finds in the picture. If the program does not find the required files it will try to generte them based on the directory structure in 'face_data/raw/'. Currently, however, this would not work, as there is no compatible folder structure present by default.

    picture = face_recognition.load_image_file(inputImage)
    face_locations = face_recognition.face_locations(picture)
    face_encodings = face_recognition.face_encodings(picture, face_locations, model = 'large')

In these lines the program loads the image it has to process and identifies the locations of the faces present in the picture and their corresponding encoding, which are stored in the variables 'face_locations' and 'face_encodings'. 'face_locations' is relatively self explanatory, 'face_encodings' is a list containing the data needed (and provided) by the library I use for the facial recognition stuff. Basicaly the library converts the facial points in the picture of each face into some number values which it then can use to check how similar two faces are. In the 'face_data/encodings.dat' file are the encodings of the politicians used in the model. Please note the extra parameter 'model = 'large'' which tells the library that it should used more points per face which makes the overall recognition part more accurate.

pil_image = Image.fromarray(picture)

With this line the program converts the previously opened picture to a format with which I can edit using pillow. This will be used to draw the rectangle and the optional face landmarks on the picture.

Speaking of face landmarks, this code executes if the optional 'showFaceLandmarks' parameter in the function is set to True:

if showFaceLandmarks:
        face_landmarks_list = face_recognition.face_landmarks(picture, face_locations, 'large')
        for face_landmarks in face_landmarks_list:
            draw = ImageDraw.Draw(pil_image)
            for facial_feature in face_landmarks.keys():
                draw.line(face_landmarks[facial_feature], width=2, fill = (255, 0, 0))
            del draw

This code is responsible for drawing these cool looking red lines connecting the points used by the library to determine how a face looks.

The following for-loop goes through each face located in the image and analyzes the current face.

matches = face_recognition.compare_faces(known_face_encodings, face_encodings, tolerance = 0.5)
similar = face_recognition.face_distance(known_face_encodings, face_encodings)

The first line determines if the currently processed face is of one of the party members in which case the program will show the name and the party the face belongs to. This is happening in the following lines

if True in matches:
            first_match_index = matches.index(True)
            information = known_face_names[first_match_index] + ' - ' + known_face_partys[first_match_index]

The second line, however, determines how similar the face is to the faces in the model.

list = similar.tolist()
lowest_match_index = list.index(min(list))
information = known_face_partys[lowest_match_index]

With this code it converts the similarity values to a list and finds the element with the lowest similarity value, as this is the one that looks the most like the face that is currently processed in the for-loop.

draw = ImageDraw.Draw(pil_image)
draw.rectangle(((left, top), (right, bottom)), outline = (0, 255, 255), width = 5)
draw.rectangle(((left, bottom), (right, bottom + 21)), fill = (0, 0, 0), outline = (0, 0, 0), width = 5)
draw.text((left + 6, bottom + 6), information, fill = (255, 255, 255))
del draw

Finally, it draws the box around the face and a rectangle below it in which it writes the name and/or the party the face corresponds to. And saves the image to the designated location and file name.

pil_image.save(outputImage)

Problems

The program does not generate an average between the party members, instead it uses the party of the member who looks most like the person in the picture.

License and credits

This work is made available under the GNU Affero General Public License v3.0.

Project made by Benjamin Hupf.

Postscript

A similar project like this was build using the self-reported political orientation of over a million participants from Facebook and dating website acconts from three countries (the U.S, the UK and Canada). This facial recognition model achived an accuracy of around 72%. If you want to read the article yourself you can find it HERE

Footnotes

1. The Wahl-O-Mat is a website by the german bpb with questions you can fill outand afterwards you can see wich political partys have similar interests. This is supposed to help people make up their mind about who they should vote for. ↩

2. https://github.com/ageitgey/face_recognition/issues/175#issue-257710508 ↩