Table of Contents
Did you know that humans should blink around 15-20 times a minute? According to Healthline, computer vision syndrome may be a common reason why most people do not blink. Ironically, this program utilizes computer vision to prevent computer vision syndrome.
Welcome to the Bleenk program!
By enabling camera, my program will count the number of times you blink and will play a sound to notify if you haven't blinked 25 times within a minute.
Run this in the background whenever you're working as a reminder to keep blinking. Don't forget to stay hydrated too!
First, we need to detect the face. To do so, we use opencv to open the camera and dlib to detect the face. Opencv produces an BGR image (blue-green-red aka true color image), so we must convert it to a grayscale image (which represents light intensity) for dlib to recognize. Once a face is detected, dlib produces 68 face landmarks around the face to detail each component. It's crucial for dlib to use the shape_predictor dab file as its reference model for it to detect face landmarks.
For this project, we are concerned about the face landmarks for the eyes (37-42 for the right eye, 43-48 for the left eye). So how do we know when the eye blinks? We can calculate its eye aspect ratio (EAR) and find when the ratio reaches a low point. In my code, I set the EAR to be 0.17, but you may change this value to your preferences.
Basically, while my program runs, it will keep a counter whenever an EAR is below a certain threshold to be considered a blink.
As stated earlier, I arbitrarily chose a constant EAR value as the threshold to close the eye. To be more accurate, I could calculate the average slope produced (visually shown in the figure/graph) and keep track of when the change in EAR is drastically slow as an indicate of a blink. This is far more accurate as an indicator of a blink isn't precisely on a constant value, but a sudden change. Furthermore, the glare from my glasses interferes with accurate eye landmarks and so it incorrectly counts more blinks than actuality.
I had so much fun creating this project. Flipping on my camera and look at my face be covered in green made me forget the hours I struggled coding. Although it wasn't necessary, I wanted to plot down the data for myself. I learned that csv files were the best way of storing sensory data. So now I have a Python file on creating a csv file and how to animate a plot in real time. It was like a second mini project on top of the eye detection.
To get a local copy up and running follow these simple example steps.
Use Python 3.9.4. Here are a list of the packages I used for the program. To install packages, please run the command:
pip install -r requirements.txt-
OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. This is necessary for enabling the camera.
-
Dlib. Necesarry for machine learning face landmark detection.
-
SciPy is a free and open-source Python library used for scientific computing and technical computing. SciPy contains modules for optimization, linear algebra, integration, interpolation, special functions, FFT, signal and image processing, ODE solvers and other tasks common in science and engineering. Necessary for calculating EAR.
-
Matplotlib is a comprehensive library for creating static, animated, and interactive visualizations in Python. Matplotlib makes easy things easy and hard things possible.
-
Pandas is a software library written for the Python programming language for data manipulation and analysis.
-
Python-csv necessary to creating and updating the csv file.
Note
If you come across other errors for installation, you may want to try some of these fixes:
pip install setuptoolspip install --upgrade wheelIf you are having trouble downloading dlib, you may try an alternative method of dlib: https://github.com/sachadee/Dlib
- Clone the repo
git clone https://github.com/clngo/bleenk.gitRun bleenk.py to start the program.
It will first provide a list of options to choose from. You can enable face landmarks, eye landmarks, or even an indicator of which eye you wink at. To turn off the program from any of these options enabled, press ESC. Otherwise, CTRL+C. The last option to enable stats will produce a graph that plots the eye aspect ratio (EAR) over time. Disabling all of the functions will not disable the program from counting the number of blinks and the cartoon blinking sound effect if the number of blinks is below 25 within 60 seconds.
Colin Ngo - https://www.linkedin.com/in/colin-ngo654 - cngo27@calpoly.edu
Project Link: https://github.com/clngo/bleenk