A simple Fingers Detection (or Gesture Recognition) using OpenCV and Python with background substraction 简单手势识别
Clone or download
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
material add README.md Oct 24, 2016
.gitignore add README.md Oct 24, 2016
README.md Update README.md May 21, 2018
new.py [!] add learningRate Jun 2, 2018

README.md

for people using python2 and opencv2, please check out the lzane:py2_opencv2 branch.

Environment

  • OS: MacOS El Capitan
  • Platform: Python 3
  • Librarys:
    • OpenCV 3
    • appscript

Demo Videos

How to run it?

  • run it in python
  • press 'b' to capture the background model (Remember to move your hand out of the blue rectangle)
  • press 'r' to reset the backgroud model
  • press 'ESC' to exit

Process

Capture original image

Capture video from camera and pick up a frame.

Alt text

Capture background model & Background subtraction

Use background subtraction method called Gaussian Mixture-based Background/Foreground Segmentation Algorithm to subtract background.

For more information about the method, check Zivkovic2004

Here I use the OpenCV's built-in function BackgroundSubtractorMOG2 to subtract background.

bgModel = cv2.BackgroundSubtractorMOG2(0, bgSubThreshold)

Build a background subtractor model

fgmask = bgModel.apply(frame)

Apply the model to a frame

res = cv2.bitwise_and(frame, frame, mask=fgmask)

Get the foreground(hand) image

Alt text

Gaussian blur & Threshold

gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)

First convert the image to gray scale.

blur = cv2.GaussianBlur(gray, (blurValue, blurValue), 0)

By Gaussian blurring, we create smooth transition from one color to another and reduce the edge content.

Alt text

ret, thresh = cv2.threshold(blur, threshold, 255, cv2.THRESH_BINARY)

We use thresholding to create binary images from grayscale images.

Alt text

Contour & Hull & Convexity

We now need to find out the hand contour from the binary image we created before and detect fingers (or in other words, recognize gestures)

contours, hierarchy = cv2.findContours(thresh1, cv2.RETR_TREE, cv2.CHAIN_APPROX_SIMPLE)

This function will find all the contours from the binary image. We need to get the biggest contours (our hand) based on their area since we can assume that our hand will be the biggest contour in this situation. (it's obvious)

After picking up our hand, we can create its hull and detect the defects by calling :

hull = cv2.convexHull(res)
defects = cv2.convexityDefects(res, hull)

Alt text

Now we have the number of fingers. How to use this information? It's based on your imagination...

I add in a keyboard simulation package named appscript as interface to control Chrome's dinosaur game.

Alt text


References & Tutorials

  1. OpenCV documentation: http://docs.opencv.org/2.4.13/
  2. Opencv python hand gesture recognition: http://creat-tabu.blogspot.com/2013/08/opencv-python-hand-gesture-recognition.html
  3. Mahaveerverma's hand gesture recognition project: hand-gesture-recognition-opencv