Machine learning paint program that uses your hand as a brush!
Note:
1. Demos (gifs) will take a bit of time to load coz they are heavy 🧸
2. You are required to install the required libraries using the requirements.txt file.
3. To run the notebooks locally you are required to install Jupyter-notebook
OR Jupyter-lab.
4. Although python2 is deprecated, If you have both python2 and python3 installed
replace python with python3 for running the application and demos.
- Python 3.7 or higher
- A working webcam
- Clone the repository:
cd <repository-directory> git clone https://github.com/lorforlinux/hand-brush.git
- Install the required libraries:
pip install -r requirements.txt
This program uses a webcam and hand gestures to control mouse movements and clicks in real time. It leverages MediaPipe for hand tracking, OpenCV for video processing, and PyAutoGUI for mouse control.
- Move Mouse Cursor: Use your index finger to control the cursor position.
- Left Click: Close your fist.
- Right Click: Open your hand and flex your fingers.
- Smooth Cursor Movement: Includes smoothing and thresholds for responsive control.
python cursor.py
This program allows users to draw on a virtual canvas using hand gestures detected via a webcam. It leverages MediaPipe for real-time hand tracking and OpenCV for rendering the canvas. Users can draw freehand, toggle straight-line mode, erase, and customize brush settings and colors.
- Freehand Drawing: Use your hand to draw lines on the canvas.
- Customizable Brush: Change brush size and color.
- Clear Canvas: Clear the entire drawing canvas with a single command.
- On-Screen Tutorial: Toggle an interactive tutorial for controls.
python handdraw.py
- q: Quit the program.
- e: Clear the canvas.
- r: Change brush color to Red.
- g: Change brush color to Green.
- b: Change brush color to Blue.
- k: Change brush color to Black.
- y: Change brush color to Cyan.
- +: Increase brush size.
- -: Decrease brush size.
- t: Show or hide the tutorial.
For hand traking my application here is using the pretrained ``SSD with MobilenetV1` model from EvilPort2's hand tracking repository. The paint toolbox uses the code from acl21's Webcam Paint OpenCV repository. Hand Brush program is truly a combination of those two repositories and i highly recommend you to check out their repositories 🦔
# for running hand brush program
python handbrush.py
Watch the demo with updated tool box here.
This demo uses EvilPort2's hand tracking model but you can also try the victordibia's hand tracking model. I tried both and got better result from the first one.
# for running hand detection demo
python handdetect.py
OR
python detect_single_threaded.py
This demo uses the akshaybahadur21's Hand Movement Tracking code for creating the pointer trail.
# for running hand pointer demo
python handpointer.py
This one is just for fun and learning How can I find contours inside ROI using opencv and Python?.
# for running contour demo
python handcontour.py
If you want to learn how does it work internally, the best way is to follow the Jupyter Notebooks i have included in this repository.
| Notebook | Purpose |
|---|---|
| detecthand.ipynb | This notebook will give you the base code for all the other notebooks and for the application itself. |
| gethand.ipynb | Here you'll learn to extract the detected hand as separate image data. |
| centeroid.ipynb | We will calculate the centeroid point in this notebook. |
| countour.ipynb | We will find the contours in the detected hand by taking it as ROI (Region Of Interest) |
If you find any issue in the code OR want any help please create an issue in the issue tracker :)