Add video support #10
Comments
|
Does OpenCV make it possible to interpolate face positions between frames? I've had several instances where I've tried to convert a series of frames, but it fails to detect faces in a few frames in between; causing a glitching effect where the target face turns on and off. (link semi-nsfw) |
|
Indeed this a common issue. Face detection is often blocked by compression artifacts. It is the case in still pics and is even more present in videos. I don't think there is a built in interpolation, but you can try doing it manually. While parsing frames, just keep a track of 1 or 2 previous frames and if current frame has nothing detected, do a 'mean' with previous data. Other possibility is to keep the result of the last detection, and keep frames without faces until you detect another face. If the new face is close to the last, apply the new detection (or the old, or an interpolation) to the buffered frames. (Don't forget to give a max size to your buffer and flush it if no new face has been detected for x frames) I'll try to tackle that later on... |
|
For now, I have an issue with input video reading. It seems python-opencv has no video decoder so I had to compile the full version... |
|
As I see on your video, there are also little glitches due to the face generator that may generate different faces for 2 consecutive frames. I don't see a clear solution here for now. |
|
I'm giving up for now, opencv-python seems not to have video support, and the full opencv lib is a pain to build.... I'll try to handle sequences of photos instead (video will just be a sequence of frames so it will adapt easily to videos) |
|
Remark: ‘python-opencv‘ seems not to have video support built-in as default. Is ‘moviepy‘ a better alternative? |
|
In the end, all these libraries boil down to is basic interaction with ffmpeg. I'm pretty sure python-OpenCV has this, but only if configured to use it. I would opt for the library with the most straightforward/pythonic interface. I would guess that is most likely moviepy, but I will look further into this after I've recovered from the new years hangover. |
|
How about taking existing code from somewhere that extracts all frames from a video, running the face swap over those and then taking the output and combine it to a video? |
|
@IcyTv Thats just writing a 4 line bash script to run the existing tools.... |
# This is the 1st commit message: remove gan_v2_2 # The commit message #2 will be skipped: # Add mask loss options to config # The commit message #3 will be skipped: # Creating Input Size config for models # # Creating Input Size config for models # # Will be used downstream in converters. # # Also name change of image_shape to input_shape to clarify ( for future models with potentially different output_shapes) # The commit message #4 will be skipped: # MTCNN options to config.ini. Remove GAN config. Update USAGE.md # The commit message #5 will be skipped: # Add sliders for numerical values in GUI # The commit message deepfakes#6 will be skipped: # Add config plugins menu to gui. Validate config # The commit message deepfakes#7 will be skipped: # Only backup model if loss has dropped. Get training working again # The commit message deepfakes#8 will be skipped: # bugfixes # The commit message deepfakes#9 will be skipped: # Standardise loss printing # The commit message deepfakes#10 will be skipped: # GUI idle cpu fixes. Graph loss fix. # The commit message deepfakes#11 will be skipped: # cleanup test code # The commit message deepfakes#12 will be skipped: # Update seg_face.py # The commit message deepfakes#13 will be skipped: # Update seg_face.py # The commit message deepfakes#14 will be skipped: # Update seg_face.py # The commit message deepfakes#15 will be skipped: # Create blur.py
I'll try to tackle that one with the simple cv2 support
The text was updated successfully, but these errors were encountered: