VisionInk

AI Hand-Tracking Drawing & Gesture Recognition — Draw in the Air

Features · Gesture Library · How It Works · Architecture · Tech Stack · Getting Started

---

VisionInk is a browser-based drawing canvas controlled entirely by hand gestures. Powered by MediaPipe Tasks Vision, it recognizes a rich vocabulary of hand poses — from drawing with one finger to erasing, zooming, panning, saving, undo/redo, and selection — turning any webcam into a digital art studio.

Everything runs client-side: no frames are uploaded, no backend required. The app is built on a clean event-driven architecture where a gesture engine, state machine, and canvas engine communicate through a central event bus.

---

✨ Features

• ✍️ Air drawing — draw with your index finger; stroke rendering with configurable color, brush size, and eraser
• 🎨 Toolbar — brush color picker, size control, erase, clear, undo/redo, save, and zoom tools
• 🧠 12+ gesture vocabulary — open palm, one-finger draw, peace sign (eraser), pinch zoom, two-finger pan, thumb-up (save), thumb-index select, closed fist, OK sign, three-finger undo, four-finger redo, thumb-down
• 🖼️ Live camera overlay — see your hand tracked in real time while you draw
• ⚡ Event-driven engine — EventBus + AppStateMachine decouple detection from action
• 📊 Live status bar — current gesture, hand count, FPS, and stroke count on screen

---

🖐️ Gesture Library

Gesture	Detector	Action
✋ Open palm	open-palm.ts	Neutral / cursor mode
☝️ One finger up	one-finger-draw.ts	Draw on canvas
✌️ Peace sign	peace-eraser.ts	Erase strokes
🤏 Pinch	pinch-zoom.ts	Zoom canvas
🤘 Two-finger	two-finger-pan.ts	Pan canvas
👍 Thumb up	thumb-up-save.ts	Save artwork
🤛 Closed fist	closed-fist.ts	Action modifier
👌 OK sign	ok-sign.ts	Confirm / select
🖖 Three-finger	three-finger-undo.ts	Undo
🖐️ Four-finger	four-finger-redo.ts	Redo
👎 Thumb down	thumb-down.ts	Cancel / reset
👆 Thumb + index	thumb-index-select.ts	Select elements

---

🕹️ How It Works

flowchart LR
    A[Webcam] --> B[MediaPipe Tasks Vision]
    B --> C{Hands detected?}
    C -->|Yes| D[Gesture Detectors]
    C -->|No| E[Idle State]
    D --> F[GestureEngine]
    F --> G[EventBus]
    G --> H[AppStateMachine]
    H --> I[CanvasEngine]
    I --> J[Draw / Erase / Zoom / Pan / Save]

Loading

1. The webcam feed is processed by MediaPipe for 21-point hand landmarks
2. Every frame, the gesture registry evaluates all detectors and emits recognized gestures
3. The state machine translates gestures into application commands
4. The canvas engine executes the command and re-renders the stroke layer

---

🏗️ System Architecture

src/
├── app/                    # Next.js app shell
├── components/
│   ├── Toolbar.tsx         # Brush, color, size, tool controls
│   └── StatusBar.tsx       # Live gesture + telemetry readout
├── drawing/
│   └── canvas-engine.ts    # Stroke rendering, layers, canvas state
├── gestures/
│   ├── registry.ts         # Gesture detection registry
│   └── detectors/          # 12 individual gesture detectors
├── hooks/
│   ├── use-hand-tracking.ts # MediaPipe landmark loop
│   └── gesture-engine.ts    # Detection → event pipeline
└── engine/
    ├── event-bus.ts        # Central pub/sub bus
    ├── state-machine.ts    # App mode state machine
    ├── events.ts           # Typed event definitions
    └── types.ts            # Domain types (AppMode, GestureType)

Design principles: gesture detection is fully decoupled from canvas rendering; new gestures are added by dropping a detector into the registry — no engine changes required.

---

🧰 Tech Stack

Layer	Technology
Framework	Next.js 16 · React 19
Language	TypeScript (strict)
Vision AI	MediaPipe Tasks Vision
Styling	Tailwind CSS 4
Rendering	HTML5 Canvas
Deployment	Vercel

---

🚀 Getting Started

Prerequisites

• Node.js 18+
• A webcam

Installation

# 1. Clone & install
git clone https://github.com/Manthan-13521/VisionInk-Hand-Tracking.git
cd VisionInk-Hand-Tracking
npm install

# 2. Run the dev server
npm run dev

# 3. Open the app
# http://localhost:3000

Scripts

Script	Description
npm run dev	Start the Next.js dev server
npm run build	Production build
npm run start	Serve the production build
npm run lint	ESLint checks

---

🔭 Roadmap

• Stroke persistence (localStorage / export PNG)
• Multi-canvas layers with gesture switching
• More gestures: shapes, fill, color gestures
• Gesture sensitivity calibration UI
• PWA offline install

---

🤝 Contributing

Contributions are welcome! Add a new gesture by implementing a detector in src/gestures/detectors/ and registering it in registry.ts.

---

📄 License

All rights reserved.

---

© 2026 Manthan Jaiswal — Built with Next.js, TypeScript & MediaPipe.

🚀 Try VisionInk