Real-time face landmark point cloud — MediaPipe × Three.js
An experimental creative-tech piece that maps 478 facial landmarks onto a living, glowing point cloud rendered in WebGL. Look into your camera and watch your face dissolve into light.
This project is actively in development — treating it as a playground for real-time computer vision and generative 3D rendering in the browser.
The browser reads your webcam stream frame-by-frame and passes each frame through Google's MediaPipe Face Landmarker model, which returns 478 3D landmark coordinates spanning the full face mesh — including 10 iris points.
Those coordinates are fed into a Three.js Points geometry each frame. Every point is rendered as an additive-blended glow sprite, so overlapping points naturally bloom. The result looks less like a tracking overlay and more like a portrait made of light.
Zone-based coloring:
- Eyes — soft cornflower blue
- Irises — bright aqua
- Lips — dusty rose
- Rest of face — depth-mapped white (closer = brighter, with a faint blue cast)
When no face is in frame, the cloud relaxes into a slowly rotating, color-cycling ellipse — an idle state that keeps the canvas alive.
| Layer | Tool |
|---|---|
| Face tracking | MediaPipe Tasks Vision 0.10.14 |
| 3D rendering | Three.js r164 |
| Build | Vite 5 |
| Shading | Custom PointsMaterial with additive blending + canvas-baked glow texture |
No frameworks. No canvas 2D. Pure WebGL point cloud.
npm install
npm run devOpen http://localhost:5173/face-cloud/ and allow camera access when prompted. The MediaPipe model loads from CDN on first run (~a few seconds).
Why points, not a mesh? A mesh would look like a 3D scan. Points dissolve the face into something more abstract — the topology is still legible but the rendering stays expressive. The glow-blending pushes it further into data-portrait territory.
Why additive blending? Dense regions (around the eyes, lips) accumulate light naturally. No manual brightness tuning needed per region — the geometry itself creates the luminance gradient.
Dual animation loops
Detection runs on requestAnimationFrame tied to video.currentTime (only processes new frames). Rendering runs on its own rAF loop and lerps positions toward targets — so the cloud moves fluidly even when detection runs slower than render.
Depth exaggeration
MediaPipe's Z values are normalized and subtle. A zScale multiplier pushes depth into readable 3D space so the cloud reads as volumetric rather than flat.
Experimental / in development. Current focus areas:
- Multi-face support
- Gesture-triggered color themes
- Blend shape expression mapping → particle physics
- Audio-reactive point sizing
Deploy guide in DEPLOY.md — covers Vercel (recommended) and GitHub Pages.
Built by Karen Dettmar