Stable Diffusion Music Videos

This is a work in progress. It is not in a usable state yet.

What is this?

This is a gradio interface for stable diffusion generated music videos. This demo is based on the stable diffusion videos project by Nathan Raw. I have added a few features to the original project to automate the process.

Features

• Download songs from Spotify
• Download lyrics from Genius
• Analyze percussive content (e.g. drums) of songs
• Turn lyrics into a set of prompts
• Generate music videos from prompts
• Generate music videos from prompts combined with percussive content (Creates the effect of a music video with lyrics that match the beat of the song)

Requirements

• Python 3.10
• Anaconda
• Genius Access Token
• Spotify Client ID and Secret

Installation

Clone

git clone ...
cd stable-diffusion-music-videos

Setup

conda env create --file env.yml
conda activate sdmv

Usage

To start the Gradio app, run:

python -m stable_diffusion_music_videos

Contributing

Setup

conda env create --file env.yml
conda activate sdmv
pre-commit install

Live Reload

gradio stable_diffusion_music_videos/app.py

Testing

pytest

Linting

flake8

Formatting

black .

License

MIT

Acknowledgements

• Gradio
• Stable Diffusion
• Stable Diffusion Videos

Contact

Created by @thehappydinoa - feel free to contact me on Twitter!