Hand Segmentation and Fingertip Tracking from Depth Camera Images Using Deep Convolutional Neural Network and Multi-task SegNet

Duong Nguyen and Soo-Hyung Kim
Chonnam National University
https://arxiv.org/abs/1901.03465

Testing Environment

• Kinect V2
• GTX 1080
• Windows 10 x64
• Anaconda 4.5, Python 3.5
• Kinect for Windows SDK v2
• PyKinect2 (https://github.com/Kinect/PyKinect2)
• Tensorflow 1.4.0
• Keras 2.0.6
• CUDA 8.0
• cuDNN v5
• OpenCV 3 for Python
  (see this installation guide https://github.com/nhduong/intro_deep#-environment-setup-1-with-gpus)

System Overview

Visual Results

Real-time Demo (Youtube videos)

Hand detection and segmentation + Fingertip detection #1
Hand detection and segmentation + Fingertip detection #2
Hand detection and segmentation + Fingertip detection #3
Hand detection and segmentation + Fingertip detection + Hand gesture recognition

Source Code and Usage

coming soon