finger-control

Real-time hand tracking demo using MediaPipe Hands.

Quick Start

Live Demo

🌐 Try it now on GitHub Pages - No installation needed!

Local Development

1. Start a local web server using one of these methods:

   # Using Node.js (if you have npx installed)
   npx http-server -p 8000
   
   # Using PHP
   php -S localhost:8000
   
   # Using VS Code Live Server extension
   # Right-click index.html > "Open with Live Server"

2. Open in browser:

   • http://localhost:8000/index.html

Features

• 🖐️ Real-time hand tracking via webcam (MediaPipe Hands)
• 📹 Live video feed with hand landmark detection
• 🎨 Visual hand skeleton overlay (green for right hand, red for left hand)
• 📊 3D hand landmark visualization
• ⚙️ Adjustable detection and tracking parameters
• 👁️ Selfie mode toggle
• 📈 FPS counter
• 🎭 NEW: Pose Animator SVG overlay - Animate custom SVG characters with hand movements!

Pose Animator Feature

The Pose Animator feature allows you to display an animated SVG overlay that follows your hand movements in real-time.

How it works

• The SVG skeleton is dynamically stretched and repositioned based on MediaPipe's 21 hand landmarks
• Each joint in the SVG corresponds to a specific hand landmark
• Bones (connections) between joints are automatically updated to follow hand movements

Usage

1. Enable Pose Animator: Toggle the "Pose Animator" switch in the control panel
2. Use Default SVG: The default hand skeleton SVG (hand-skeleton.svg) is loaded automatically
3. Load Custom SVG:
   • Click "Choose File" under "Load Custom SVG"
   • Select your custom SVG file
   • The SVG will be animated using your hand landmarks

Creating Custom SVG for Pose Animator

To create your own animated SVG character, follow these guidelines:

1. Use Landmark IDs: Name your SVG elements with IDs that correspond to MediaPipe hand landmarks:

   • Joints (circles): joint-0 through joint-20 (where 0 is wrist, 1-4 is thumb, 5-8 is index, etc.)
   • Bones (lines): palm-0-1, thumb-1-2, index-5-6, etc.

2. Hand Landmark Structure (21 points):

   0: Wrist
   1-4: Thumb (base to tip)
   5-8: Index finger (base to tip)
   9-12: Middle finger (base to tip)
   13-16: Ring finger (base to tip)
   17-20: Pinky finger (base to tip)
   

3. Example SVG Structure:

   <svg id="hand-character" viewBox="0 0 800 800">
     <g id="skeleton">
       <line id="thumb-1-2" class="bone" />
       <circle id="joint-1" class="joint" />
       <!-- Add more joints and bones -->
     </g>
   </svg>

4. Reference: See hand-skeleton.svg for a complete example

Technical Details

Based on the Pose Animator technique from TensorFlow:

• Blog post: https://blog.tensorflow.org/2020/05/pose-animator-open-source-tool-to-bring-svg-characters-to-life.html
• Reference repo: https://github.com/yemount/pose-animator

The implementation:

• Converts MediaPipe normalized coordinates (0-1) to canvas coordinates
• Updates SVG element positions in real-time
• Renders the animated SVG overlay on the canvas
• Maintains compatibility with all existing features

Requirements

• Modern web browser (Chrome recommended)
• Webcam
• Internet connection (for CDN libraries)

How It Works

This demo uses MediaPipe Hands to detect and track hand landmarks in real-time from your webcam feed. The application:

1. Captures video from your webcam
2. Processes each frame through MediaPipe's hand detection model
3. Draws the detected hand landmarks and connections on the canvas
4. Optionally animates an SVG overlay using the detected landmarks (Pose Animator mode)

Based on the MediaPipe Hands JavaScript demo.