Permalink
Switch branches/tags
Nothing to show
Find file Copy path
Fetching contributors…
Cannot retrieve contributors at this time
29 lines (15 sloc) 1.32 KB

neural-style-video

Application of Gatys' neural style transfer algorithm to video.

Much of the following is lifted from Gene Kogans instructions.

  1. Extract frames from video using ffmpeg

    ffmpeg -i myMovie.mp4 -r 12 -f image2 image-%5d.jpg ffmpeg -i myMovie.mp4 rawaudio.wav

This will extract frames at 12 fps.

  1. Install and run jcjohnsons implementation of the neural style transfer algorithm, with dependencies.

  2. Run the neural style transfer algorithm.

    th neural_style.lua -style_image myStyle.jpg -content_image image-00001.jpg -output_image generated-00001.jpg

  3. Calculate optical flow, morph and blend in the stylized image.

    python opticalflow.py --input_1 image-00001.jpg --input_1_styled generated-00001.jpg --input_2 image-00002.jpg --output blended-00001.jpg --alpha 0.05 --flowblend False

(The flowblend option will set scale the opacity according to the amount of motion - I didn't get good results with this, so you should probably set it to false)

  1. When all frames are blended, you can merge them back into a video by calling

    ffmpeg -framerate 12 -i generated-%05d.png -i rawaudio.wav -c:v libx264 -preset veryslow -qp 0 -pix_fmt yuv420p myMovie.mp4