Gesture AR System

Control your computer with hand gestures using computer vision and machine learning.

Features

• Hand Tracking - Real-time 21-point hand landmark detection using MediaPipe
• Gesture Recognition - Detect pinch, fist, pointing, open palm, peace sign, thumbs up
• Cursor Control - Move mouse, click, drag, and scroll with hand gestures
• Air Drawing - Draw in the air with your finger and save your artwork
• Voice Commands - Optional voice control for hands-free actions
• Kalman Filtering - Smooth, jitter-free cursor movement

Demo

Gesture	Action (Cursor Mode)	Action (Draw Mode)
Point (Index finger)	Move cursor	Draw line
Pinch (Thumb + Index)	Click	Select color
Fist	Drag	Stop drawing
Open Palm	Release	Stop drawing
Peace Sign	Double click	-

Requirements

• Python 3.8 or higher
• Webcam
• Windows, macOS, or Linux

Installation

1. Clone the repository

```bash git clone https://github.com/rachts/Air-mouse.git cd gesture-ar-system ```

2. Create a virtual environment (recommended)

```bash python -m venv venv

Windows

venv\Scripts\activate

macOS/Linux

source venv/bin/activate ```

3. Install dependencies

```bash pip install -r scripts/requirements.txt ```

4. Install voice commands (optional)

Voice commands require PyAudio which needs additional setup:

Windows: ```bash pip install pyaudio ```

macOS: ```bash brew install portaudio pip install pyaudio ```

Linux (Ubuntu/Debian): ```bash sudo apt-get install python3-pyaudio portaudio19-dev pip install pyaudio ```

Usage

Quick Start

```bash cd scripts python main.py ```

Command Line Options

```bash python main.py --camera 0 # Use camera index 0 (default) python main.py --camera 1 # Use external camera python main.py --no-cursor # Disable cursor control python main.py --no-draw # Disable drawing mode python main.py --voice # Enable voice commands python main.py --no-mirror # Disable mirror mode ```

Keyboard Controls

Key	Action
Q / ESC	Quit
M	Toggle mode (cursor/draw)
L	Toggle landmark display
C	Clear drawing
S	Save drawing to file

Voice Commands

When enabled with --voice:

• Actions: "click", "double click", "drag", "drop"
• Drawing: "clear", "undo"
• Colors: "red", "blue", "green", "yellow", "purple", "white"
• Modes: "cursor", "draw"

Project Structure

``` gesture-ar-system/ ├── scripts/ │ ├── main.py # Main application │ ├── hand_tracker.py # MediaPipe hand tracking │ ├── gestures.py # Gesture recognition │ ├── cursor_controller.py # Mouse control │ ├── kalman_filter.py # Smoothing filter │ ├── air_draw.py # Air drawing │ ├── voice_commands.py # Voice recognition │ └── requirements.txt # Dependencies └── README.md ```

Troubleshooting

Camera not detected

```bash python main.py --camera 1 python main.py --camera 2 ```

MediaPipe errors

```bash pip uninstall mediapipe pip install mediapipe ```

PyAutoGUI permission errors (macOS)

Go to System Preferences > Security & Privacy > Privacy > Accessibility and add Terminal or your Python IDE.

PyAudio installation fails

• Windows: pip install pipwin && pipwin install pyaudio
• macOS: Install portaudio first with brew install portaudio
• Linux: Install portaudio19-dev before pip install

Tips for Best Results

1. Use good, even lighting on your hands
2. Plain backgrounds work better than cluttered ones
3. Keep your hand 1-3 feet from the camera
4. Move your hand slowly for better tracking
5. Make clear, distinct gestures

Contributing

1. Fork the repository
2. Create your feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• MediaPipe for hand tracking
• OpenCV for computer vision
• PyAutoGUI for cursor control