An Iron Man–inspired holographic orb built with Next.js, Three.js, and MediaPipe hand tracking — control it with your bare hands through your webcam.
🔮 This is the open-source interface of ULTRON — my AI that talks in real time and controls Android devices by itself. Read the write-up or the X post
demo.mp4
npm install
npm run devOpen http://localhost:3000.
| Input | Action |
|---|---|
| Drag | Spin the orb |
| Scroll / pinch | Zoom in & out |
Click GESTURES OFF (or press G) and allow camera access, then:
| Gesture | Action |
|---|---|
| Pinch (thumb + index) one hand and move it | Spin the orb |
| Pinch with both hands, spread apart / bring together | Zoom in / out |
| Key | Action |
|---|---|
G |
Toggle hand gestures |
R |
Reset the view |
+ / − |
Zoom in / out |
lib/orbScene.ts— the Three.js scene: layered wireframe shells, a spiral inner core, floating code-text sprites, orbiting debris, dust particles, scan rings, and a bloom + chromatic-aberration post-processing stack.lib/handTracker.ts— MediaPipe HandLandmarker running on the webcam feed. Pinch detection with hysteresis: one pinched hand spins the orb, two pinched hands zoom by spreading apart or together.components/JarvisOrb.tsx— the HUD and glue between the scene, the tracker, and your inputs.
MIT