A real-time music visualizer built for live performance. Point it at an audio source and it generates a self-directed cinematic journey — scenes zoom into each other, worlds unfold from worlds, and the visual story responds to every beat, bass hit, and tension shift in the music.
Built for large screens, projectors, and stages.
The visualizer listens to audio in real-time and maps it to a spatial scale story:
sparse → cosmic scale — galaxies, nebulae, black holes
building → approach — cities from above, ocean, approaching entity
peak → intimate — the humanoid figure, murmuration flocks, sacred geometry
drop → explosion — gravity collapses, screen-filling flash
falling → ethereal — drifting back out through the cosmos
When the audio phase changes, the Conductor picks a new scene from that phase's pool and triggers a cinematic transition — zoom in, zoom out, dissolve, shatter, or warp. Every cut feels motivated because it follows the scale axis.
pnpm install
pnpm devOpen http://localhost:5173. Click anywhere to start the audio context, then play music through your system audio or microphone.
pnpm build # production build
pnpm test # run unit tests
pnpm preview # preview production buildRequirements: Browser with WebGL2 support (Chrome, Firefox, Safari 15+).
| Scene | Technique | Description |
|---|---|---|
galaxy |
GPU particles (150k) | Spiral galaxy with two logarithmic arms and differential rotation — inner particles orbit faster |
black_hole |
Fullscreen raymarching | Gravitational lensing, Einstein ring, accretion disk with heat gradient |
nebula |
Volumetric raymarching | 8-sample ray accumulation through 3D fbm gas cloud |
planet |
Raymarched SDF | Sphere with fbm terrain, crater SDFs, Mie atmosphere glow, orbiting camera |
solar_system |
Three.js geometry | 6 planets orbiting a star; 2 outer planets have rings |
star_field |
GPU particles | Classic procedural star field |
terrain |
Fullscreen shader | Procedural terrain flyover |
| Scene | Technique | Description |
|---|---|---|
approaching_entity |
Fullscreen SDF | Abstract 6-tendril cosmic organism that slowly approaches, lingers at closest point, retreats — 40s cycle |
city_grid |
Fullscreen shader | Night city from above; camera descends over time; buildings with blinking windows; beat flashes districts |
wireframe_city |
Three.js LineSegments | 400 Tron-style wireframe buildings in one draw call; camera walks through at street level |
ocean |
Fullscreen shader | Gerstner wave ocean with analytically computed normals, moonlight streak, specular, whitecaps |
crystal_cave |
Fullscreen shader | Crystal cave environment |
| Scene | Technique | Description |
|---|---|---|
cosmos_figure |
Fullscreen SDF | Humanoid figure floating in a rich nebula — orbital particle halo, procedural star field, volumetric background |
murmuration |
GPGPU | 10,000 particles — ping-pong float textures, velocity compute + position integrate each frame, boid flocking rules |
sacred_geom |
Three.js geometry | Icosahedron wireframe + inner structure at golden ratio (0.618×), counter-rotating layers |
dna_helix |
InstancedMesh | Double helix — 120 sphere instances + 6 cross-connecting rungs; tension controls helix pitch |
ribbon_flow |
Fullscreen shader | Flowing ribbon forms |
figure |
Fullscreen SDF | SDF humanoid |
| Scene | Technique | Description |
|---|---|---|
gravity_well |
Vertex deformation | 120×120 plane pulled by 3 orbiting mass points via vertex shader |
particle_universe |
GPU particles (15k) | High-density particle field |
| Scene | Technique | Description |
|---|---|---|
cell_forms |
Fullscreen raymarching | 6 SDF metaballs on Lissajous paths, polynomial smin() blending |
morphing_entity |
Three.js geometry | Abstract morphing 3D entity |
| Scene | Technique | Description |
|---|---|---|
god_rays |
Fullscreen shader | 16-sample volumetric light shafts — marches toward a drifting light source, fbm noise for cloud shadow |
energy_field |
Fullscreen shader | 8-step curl-noise streamline field, slow field rotation, tension controls turbulence |
plasma |
Fullscreen shader | Plasma waves |
aurora |
Fullscreen shader | Aurora borealis |
wormhole |
Fullscreen shader | Rotating wormhole |
tunnel |
Fullscreen shader | Tunnel drift |
fractal |
Fullscreen shader | Fractal patterns |
lissajous |
Fullscreen shader | Lissajous curves |
kaleidoscope |
Fullscreen shader | Kaleidoscope |
ripple |
Fullscreen shader | Ripple effect |
Audio Input (Web Audio API + Meyda)
│
▼
AudioAnalyzer
bands: { bass, mid, high }
beatImpulse, tension, intensity
│
▼
Conductor ── PHASE_TARGETS ── TRANSITION_MAP
picks scene pair + palette based on audio phase
│
▼
LayerCompositor (React Three Fiber, WebGL2)
┌─────────────┬─────────────┐
│ backFBO │ midFBO │ offline scene rendering
└─────────────┴─────────────┘
│ snapshotFBO + tempBackFBO (transition state machine)
▼
compositor quad → screen
Every back-layer scene change triggers a cinematic transition shader. The outgoing frame is snapshotted into an FBO; the incoming scene renders into a temp FBO. A transition shader blends between them over ~1.5s (750ms on drop):
| Type | Description |
|---|---|
zoom_in |
Old scene rushes forward into center, new scene zooms out |
zoom_out |
Old scene expands to edges, new scene emerges from center |
dissolve |
Per-pixel random threshold — pixels scatter at their threshold then resolve |
shatter |
Voronoi cracks spread from a point, cells displace outward |
warp |
Domain-warp UV distortion peaks at midpoint while cross-fading |
Transitions are picked from a TRANSITION_MAP keyed by "fromScene->toScene". Unspecified pairs fall back to warp.
Every scene receives a SceneProps object each frame:
| Prop | Type | Description |
|---|---|---|
bands.bass |
number |
Low frequency energy 0–1 |
bands.mid |
number |
Mid frequency energy 0–1 |
bands.high |
number |
High frequency energy 0–1 |
beatImpulse |
number |
Spike at detected beats |
tension |
number |
Slow-building pre-drop energy |
intensity |
number |
Overall loudness |
palette |
[string, string, string] |
3 hex colors: dark / mid / accent |
These wire into GLSL uniforms or Three.js properties directly — no intermediate abstraction layer.
src/
├── engine/
│ └── narrative/
│ ├── Conductor.ts # scene selection, transitions, phase pools
│ └── __tests__/
├── scenes/
│ ├── LayerCompositor.tsx # FBO compositor + transition state machine
│ ├── SceneRenderer.tsx
│ ├── murmuration/ # GPGPU boids (4 shaders + component)
│ ├── galaxy/ # 150k GPU particle spiral
│ ├── cosmos_figure/ # SDF humanoid in nebula
│ ├── gravity_well/ # vertex-deformed mesh
│ ├── dna_helix/ # InstancedMesh double helix
│ ├── sacred_geom/ # wireframe icosahedron
│ ├── cell_forms/ # metaball raymarching
│ ├── god_rays/ # volumetric light shafts
│ ├── energy_field/ # curl-noise field lines
│ └── ... (20+ more scenes)
├── shaders/
│ ├── utils.glsl # _hash, _fbm, domainWarp
│ ├── trans_zoom_in.frag.glsl # transition shaders (5 types)
│ ├── trans_zoom_out.frag.glsl
│ ├── trans_dissolve.frag.glsl
│ ├── trans_shatter.frag.glsl
│ ├── trans_warp.frag.glsl
│ ├── composite.frag.glsl # final back+mid compositor
│ ├── murmuration_compute.frag.glsl # boid velocity compute pass
│ ├── murmuration.vert.glsl # reads position from GPU texture
│ ├── gravity_well.vert.glsl # mass-point vertex deformation
│ └── ... (35+ more shaders)
└── types/
├── audio.ts # SceneName, AudioBands, SceneProps
├── composition.ts # CompositionConfig, TransitionType
└── narrative.ts # AudioPhase
| Renderer | Three.js 0.170 + react-three-fiber v8 |
| Language | TypeScript 5.8 strict mode |
| UI framework | React 18 |
| Shaders | GLSL via vite-plugin-glsl (#include support) |
| Audio analysis | Meyda (Web Audio API) |
| Styling | Tailwind CSS v4 |
| Build | Vite 6 |
| Tests | Vitest |
| Package manager | pnpm |
- murmuration runs 2 full GPU compute passes per frame. On low-end GPUs it may drop below 60fps.
- cell_forms does 64 raymarching steps + 6 normal estimation SDF evaluations per pixel — the most expensive fullscreen scene.
- gravity_well deforms ~15k vertices per frame in the vertex shader — fast on all desktop GPUs.
- All scenes use
depthWrite: false; no shadow maps or heavy lighting. - A 15k background particle layer is always active regardless of the current scene.
| Palette | Phases |
|---|---|
midnight |
sparse, building |
ash |
sparse |
ice |
sparse |
amber |
building, falling |
crimson |
building, falling |
neon |
peak, drop |
acid |
peak |
solar |
peak, drop |
plasma |
peak, falling |
prism |
drop |
Each palette provides 3 colors mapped to u_color0 (dark/background), u_color1 (mid), u_color2 (accent/bright) in every shader.
MIT