Application of Gaty's neural style transfer algorithm to video
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information. Added link Dec 18, 2015 Initial commit Dec 18, 2015


Application of Gatys' neural style transfer algorithm to video.

Much of the following is lifted from Gene Kogans instructions.

  1. Extract frames from video using ffmpeg

    ffmpeg -i myMovie.mp4 -r 12 -f image2 image-%5d.jpg ffmpeg -i myMovie.mp4 rawaudio.wav

This will extract frames at 12 fps.

  1. Install and run jcjohnsons implementation of the neural style transfer algorithm, with dependencies.

  2. Run the neural style transfer algorithm.

    th neural_style.lua -style_image myStyle.jpg -content_image image-00001.jpg -output_image generated-00001.jpg

  3. Calculate optical flow, morph and blend in the stylized image.

    python --input_1 image-00001.jpg --input_1_styled generated-00001.jpg --input_2 image-00002.jpg --output blended-00001.jpg --alpha 0.05 --flowblend False

(The flowblend option will set scale the opacity according to the amount of motion - I didn't get good results with this, so you should probably set it to false)

  1. When all frames are blended, you can merge them back into a video by calling

    ffmpeg -framerate 12 -i generated-%05d.png -i rawaudio.wav -c:v libx264 -preset veryslow -qp 0 -pix_fmt yuv420p myMovie.mp4