Apply part swapping on selected part of full size video, like DeepFake work?

[urmit-space]

How can apply part swapping in this video on a particular portion.?
I get a very unwanted result in the whole video. I know in this video we should detect the face in the full-size video then apply the mask on a particular part. maybe, I know the process but don't know where to start. or is that use different method can you please tell that. can you please take time and help with that!!!?

ir_01.mp4

Iron man replace with this vin image

[AliaksandrSiarohin]

This is rather complicated, need to write code for frame detection and back Inpainting.

[AliaksandrSiarohin]

I guess it will be easier to try some exiting solution for this.

[urmit-space]

This is rather complicated, need to write code for frame detection and back Inpainting.

yes, it takes more time like 4-5 min to create a face-swapping. but get a good result.
can you please help me out as you say the existing solution is easier? I don't know where to change it. if you tried before, let me give some help in the solution.

[AliaksandrSiarohin]

1. Slice the video into chunks using https://github.com/AliaksandrSiarohin/first-order-model/blob/master/crop-video.py
2. Also crop image Win Diesel using paint to make it a square image.
3. Run a face swap for all chunks from 1.
4. Write a script that take modified chunks and place them back to the original positions (positions is a part of the ffmpeg command).

[urmit-space]

OK let me try it.
in this, we have to first detect the face in the video. divided video in chunks where a face appeared then crop the face portion.
after these things applied by reverse sequence. is that correct?
crop-video.py has face crop logic or have to do it myself.

[AliaksandrSiarohin]

Yes, crop video will produce you ffmpeg commands for cropping. Also you may need to tweek default parameters of crop-video, if you want to include all the chunks.

[urmit-space]

Ok, let me try once. then get to know what challenges are made from my side. will update on this soon.

[urmit-space]

I get success in this now all things work. here I attach a video. greatee bro!!

completed.1.mp4

but two things I want to change. how can make all these things automatic just one click and want to improve the result?
and other I want to change crop box size like want exectly face crop so may result will improve.
The last one how can we achieve this type of face swapping. first-order-model-Face-swap GiF
I tried many times but not get exactly this type.

[Hpjhpjhs]

@urmit-space Hi, could you share the code about how to place the croped video back to the original positions (positions is a part of the ffmpeg command). It is appreciated If I can received your reply.

[urmit-space]

@Hpjhpjhs you get x, y, and size from the crop frame. you need to add an overlay video on the current video with respect frame.

!ffmpeg -i /content/Data/4.1_original_conan_obrien.mp4 -vf "movie=/content/Data/predictionscrop.mp4, scale=429: -1 [inner]; [in][inner] overlay =736: 130 [out]" /content/Data/completed.mp4

Maybe this will help.

[Hpjhpjhs]

@urmit-space Thank you so much. That's very helpful.