A real-time hands-free mouse control application that uses face and eye tracking through a webcam. The cursor is moved using nose position, while left and right mouse clicks are performed through eye blink detection. Built with Python, MediaPipe, OpenCV, PyQt5, and PyAutoGUI for assistive technology and human-computer interaction use cases.
Eye Control Mouse
Overview
Eye Control Mouse is a real time hands free mouse control system built using Python. The application allows users to control the mouse cursor using facial movements and perform mouse clicks using eye blinks. It is designed primarily for accessibility, assistive technology, and experimental human computer interaction.
The system uses a webcam to track the user’s face and eyes. Cursor movement is mapped to the nose position, while eye blink detection is used to perform left and right mouse clicks.
File Information
The entire application logic is contained in a single Python file named EyeControlMouseFinalV4.py.
Technologies Used
Python is used as the core programming language. MediaPipe is used for face and eye landmark detection. OpenCV is used for webcam input and real time image processing. PyAutoGUI is used to control mouse movement and clicks. TensorFlow is used internally by MediaPipe for inference.
Features
Real time face and eye tracking using a USB or built in webcam. Mouse cursor movement controlled by nose position. Left click triggered using left eye blink detection. Right click triggered using right eye blink detection. Hands free interaction suitable for accessibility use cases. Low latency performance on consumer hardware.
System Requirements
Python version 3.8 to 3.11 is recommended. A working USB or built in webcam is required. Windows operating system is recommended for best compatibility.
Required Python Libraries
OpenCV MediaPipe PyAutoGUI NumPy
Installation
Install Python and ensure it is added to system PATH. Install the required dependencies using pip.
pip install opencv-python mediapipe pyautogui numpy
How to Run
Connect your webcam and ensure it is detected by the system. Open a terminal or command prompt in the project directory. Run the Python file using the following command.
python EyeControlMouseFinalV4.py
Once the application starts, the webcam window will open and mouse control will activate automatically.
Usage Instructions
Keep your face clearly visible in front of the camera. Move your nose slightly to control the mouse cursor position. Blink your left eye to perform a left mouse click. Blink your right eye to perform a right mouse click. Ensure proper lighting for better tracking accuracy.
Common Issues
If the camera does not open, ensure no other application is using the webcam. If you see a camera index error, try changing the camera index in the code. If MediaPipe errors occur, verify the installed version is compatible with your Python version.
Use Cases
Assistive technology for users with limited hand mobility. Hands free computer interaction experiments. Research and learning in computer vision and human computer interaction.
Future Improvements
Adding calibration for personalized sensitivity. Support for scroll gestures using facial movements. Cross platform compatibility improvements. GUI based controls for configuration.
License
This project is open for educational and experimental use. You are free to modify and extend it as needed.