Lucid Sonic Dreams syncs GAN-generated visuals to music. By default, it uses NVLabs StyleGAN2, with pre-trained models lifted from Justin Pinkney's consolidated repository. Custom weights and other GAN architectures can be used as well.
Sample output can be found on YouTube and Instagram.
This implementation has been teston on Python 3.6 and 3.7. As per NVLabs' TensorFlow implementation of StyleGAN2, TensorFlow 1.15 is required. TensorFlow 2.x is not supported.
To install, simply run:
pip install lucidsonicdreams
You may refer to the Lucid Sonic Dreams Tutorial Notebook for full parameter descriptions and sample code templates. A basic visualization snippet is also found below.
from lucidsonicdreams import LucidSonicDream
L = LucidSonicDream(song = 'song.mp3',
style = 'abstract photos')
L.hallucinate(file_name = 'song.mp4')