Plugin to merge frame to video with audio

[ruah1984]

HI ,
can we add a plugin to merge frame to video with audio?

[Clorr]

Hello,

I don't understand the request. Can you provide more info please ?

[Jack29913]

He asks a plugin which converts frames into the video and adds original audio from source video into fake video.

[Clorr]

:-D Frame, not flame!

I currently have in mind the @shaoanlu method with moviepy, but I have not much time to work on it. There is a draft pull request if you want to have a look

[Jack29913]

In my view this can be archived with FFMPEG easily. So it's not a priority. I'm waiting for GAN with masking it will be a game changer if its like in those gifs

[Clorr]

Can you provide some guidelines to @ruah1984 , if he needs some ?

[Clorr]

For the gan plugin, the classes are somewhat ready in https://github.com/shaoanlu/faceswap-GAN/blob/master/temp/faceswap_GAN_keras.ipynb I have to merge them, hopefully later today

[ruah1984]

sorry for the typo.
no hurry i think i can wait until GAN plugin ready in deepfakes/faceswap.
thanks.

[Jack29913]

sure. i use these commands.

frames to video
ffmpeg -f image2 -framerate <fps_of_source_video> -i "img (%d).jpg" -vcodec libx264 -profile:v high444 -refs 16 -crf 0 -preset ultrafast input_0.mp4

adding sound
ffmpeg -i input_0.mp4 -i source.mp4 -c copy -map 0:0 -map 1:1 -shortest out.mp4

before using these commands, you should copy source frames to merged folder and say NO when it asks if it should overwrite existing files. You should do thing step because there might be some frames without faces in them or algorithm may have failed to detect faces for specific frames. If you fail to do this step correctly final video and audio won't sync.

[Clorr]

I'm closing this as I consider it to be handled in #10