HandBox

A computer-vision toy that turns your two hands into a live video effects window. Hold up both index fingers, form a box between them, and the region inside gets an effect applied in real time — no mouse, no keyboard, just your hands.

Demo

Watch HandBox-Preview.mp4

Features

• Two-hand box tracking — the box is defined live by the position of your two index fingertips, tracked via MediaPipe's hand landmark model
• Real-time effects applied only inside the box:
  • Invert — classic photo-negative effect
  • Pixelate — color-quantized, blocky mosaic effect
• Gesture-based switching — pinch your two index fingertips together to cycle through effects; no keyboard required
• Smooth, stable tracking — coordinates are smoothed frame-to-frame and persist briefly through momentary tracking loss, so the box doesn't flicker
• Auto-centered window on launch

Tech Stack

• Python
• OpenCV — video capture, image processing, rendering
• MediaPipe (Tasks API — HandLandmarker, VIDEO running mode) — real-time hand landmark detection

Setup

git clone https://github.com/Versh-03/HandBox-Python.git
cd HandBox

python -m venv venv
source venv/bin/activate
pip install opencv-python mediapipe

wget -O hand_landmarker.task https://storage.googleapis.com/mediapipe-models/hand_landmarker/hand_landmarker/float16/1/hand_landmarker.task

python main.py

Usage

1. Run python main.py — a webcam window opens, centered on screen.
2. Hold up both hands in view of the camera.
3. Move your index fingers to form a box — whatever falls inside gets the active effect applied.
4. Pinch your two index fingertips together to cycle: Normal → Invert → Pixelate → Normal.
5. Press q to quit.

How It Works

Each frame, MediaPipe detects both hands and extracts the index fingertip (landmark 8) from each. Those two points define opposite corners of a bounding box, which is smoothed using an exponential moving average to reduce jitter and held briefly during momentary tracking dropouts. The pixel region inside the box is cropped, processed with the currently selected effect, and written back into the frame before display. A simple Euclidean-distance check between the two fingertips detects a pinch gesture, which cycles the active effect with a debounce lock to prevent rapid re-triggering.