The process:
-
Use ffmpeg to split videos to frames (bitmaps).
-
Loop through frames.
-
Discrete Fourier Transform (with fftw) on respective time interval for the wavefile.
-
Filter and output new frames corresponding to the audio analysis.
-
Use ffmpeg to encode an mp4 with the respective audio.
Project with end goal of generating music videos
Examples:
R. Conner Howell 2016