An all-in-one solution for unlocking a door with two-factor facial recognition of Facebook users.
All image manipulation is done with OpenCV 2. A list of Facebook users is maintained, and a model is trained to recognize the face of each user.
Authentication is granted via Facebook's Device Authorization API, and a configurable number of photos are fetched.
The photos are first cropped to roughly the area defined by the tag on Facebook (exact dimensions are not provided in the API, only the center of the tag), so as to eliminate other people. A couple Haar classifier trained on frontal faces are then used to identify the face of the user, and the image is again cropped. The eyes of this face are then identified, and used to rotate and further crop the image so that only the aligned face remains. Finally, the image is normalized by equalizing its histogram, removing any noise from lighting and contrast differences.
Online learning is supported by tracking the correctness of predictions, and re-training the model periodically.
An iOS app is used for two-factor authentication. If the phone belonging to the recognized face is within a configurable distance from the door, the door is automatically unlocked. Otherwise, a push notification is sent to verify the user's identity.
The app is also used to collect feedback in order to continuously update the facial recognition model.
- User logs in with Facebook on the iOS App
- App sends Facebook access token to server
- Server uses access token to determine the user's Facebook ID, and checks whether they are on the whitelist of users
- The server then uses the Duo Auth API to determine enrollment status for that user (Facebook ID as username)
- If the device is not enrolled, an activation code is returned, and sent back to the device
- The iOS App uses the Duo Mobile SDK to activate and enroll the device
- User is recognized by the images streamed to the server
- Server uses Duo Auth API to ensure user is enrolled, then creates an authorization request
- iOS app receives push, uses Duo Mobile SDK to get transaction info, and presents a notification to the user
- iOS app uses Duo Mobile SDK to confirm or reject the authorization
- If confirmed, door is unlocked
- If rejected, the user is taken to the app, where the photo is presented (for verification and online learning)
A Raspberry Pi is used to communicate with the server running the above code.