Demo unity scenes for dynamic mesh deformations using audio source descriptor signals. Long form demo videos are available on this YouTube playlist. This project uses the following Sonosthesia dependencies:
- com.sonosthesia.processing
- com.sonosthesia.signal
- com.sonosthesia.flow
- com.sonosthesia.target
- com.sonosthesia.mapping
- com.sonosthesia.noise
- com.sonosthesia.mesh
- com.sonosthesia.deform
- com.sonosthesia.audio
A Unity timeline is used to play audio files through audio sources. The energy in different spectral bins is extracted and fed into signals which are mapped to targets controlling mesh deformation. The deformations are implemented using the Unity Job System and Burst compiler, following Catlike coding tutorials (see the mesh and noise packages which contain the MIT License from the author). Dynamic motion is obtained by lerping through different noise generations seeds and different noise components are controlled by different frequency bands.
Audio for these demo scenes was obtained from epidemicsound, namely jonquilla and t1000.
NOTE : on the first Unity editor play the audio and the graphics are out of sync, due to Unity generating runtime support files. This issue solves itself on second playback.
This scene provides simple level bars to demonstrate control signals originating from timeline audio spectral analysis.
Simplex noise with 4 different components each lerping 3 seeds on an icosphere with 2500 vertices. 75 fps on Apple M1 Pro. Having a single noise types allows a more efficient Job System dispatch.
Heterogeneous noise (Simplex, Voronoi and Perlin) with 4 different components each lerping 3 seeds on an icosphere with 2500 vertices. 63 fps on Apple M1 Pro.
Multichannel sound visualization using shader graph. Sine waves in polar coordinates with dynamic amplitude, phase and color intensity.
Multichannel sound visualization using shader graph. Sine waves in cartesian coordinates with dynamic amplitude, phase and color intensity.
This scene combines job system based mesh deformations with shader graphs and vfx graphs controlled via sound descriptors with more sophisticated composition and flow elements from the Sonosthesia packages. Video with sound available here.
This scene combines job system based mesh generation following torus knot parametric curves with shader graphs and vfx graphs controlled via sound descriptors and merged with manual animations. Video with sound available here.
