hand_gesture_detection

Hand gesture detection based on the hand module of Google's mediapipe API. The hand module gives the coordinates of 21 hand landmarks, which can be found in the image below.

This project focuses on three functionalities:

1. Hand detection.
2. Hand gesture detection.
3. Volume control using hand gestures.

Requirements

Python 3.8 or later with dependencies listed in requirements.txt. To install run:

$ git clone https://github.com/jhan15/hand_gesture_detection.git
$ cd hand_gesture_detection
$ pip install -r requirements.txt

Usage

# Hand detector
$ python3 hand.py --max_hands 2

# Gesture detector
$ python3 gesture.py --mode single # currenly only single-hand gestures are supported

# Volume controller
$ python3 vol_controller.py --control continuous # continuous control
                                      step # step control

Demo

Hand detector

Detect hands on streams, it draws the landmarks on detected hands and returns several hand features, including handedness, landmark coordinates, hand direction, hand facing, boundary landmarks, wrist angle.

Gesture detector

Detect hand gestures on streams, now it can detect 18 pre-defined hand gestures, including Chinese number 1 - 10, 'OK', 'Thumbs-up', 'Thumbs-down', 'Rock', 'Claw', 'C shape', 'O shape', 'Pinch'.

Volume controller

Control volume using hand gestures. It's a possible application in areas like smart home and in-cabin interaction. Hand gestures can be used for activation, control, and deactivation. In this case, we use 'Pinch' as activation and control gesture and 'C shape' as deactivation gesture. There are two types of control behavior here, continuous control and step control.

Continuous control

Step control

Issue

The package I used to control Macbook's volume is osascript, it will reduce the FPS from ~30 to ~5.