-
Notifications
You must be signed in to change notification settings - Fork 2.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mouth Position Delay / Lag when motion is faster #46
Comments
Hello, thank you for the clear and detailed description of the issue. I think I clearly understand the issue. It is caused by this line: Line 97 in 0600d0f
This line smooths bounding boxes over a 5 frame window. This is desirable as face detection can be at times be noisy and change a lot across frames. But, when there is rapid motion, the face position changes so much that the averaging the box location across the frames is undesirable. We will look into this issue and update you once there is some sort of fix. |
Can you pull the latest commit, and try with the |
Just to be clear: I'm a newbie when it comes to GitHub and I'm not a programmer so everything is very new to me, all I can do is to follow instructions and tutorials. I've downloaded "inference.py" made a backup of the old one, and put this on the main master directory. Then I used the same files I did for the test of this issue and at the end I've added but I get this error: I double checked that I put the NEWEST file (from about 30 minutes ago) instead of the old one, but still it seems like it's not recognizing the new command. Maybe I did something wrong, not sure if I'm the right man to help testing but I can try if it's something simple enough for me to follow, sorry for the mess. |
Hey, I had a similar issue. I ran it with the
I see that line 101 uses boxes Line 101 in 8549b79
which we only retrieve if line 100 is true otherwise we don't set it to anything. Line 100 in 8549b79
|
@AlonDan can you check now if it is better? |
Sure, I just tried it now it won't start running there is an error right after I press ENTER:
Even if I just run the main command: I get the same error. |
Hello, I think this is an error from your end, as you can see that the repo does not have that in line 6: Line 6 in 11a8eac
|
Hmm, if that's the case I'm not sure how to fix that as I explained unfortunatly I'm a newbie in this. f I switch back to the backup (old file version before the issue) everything works, but of course without the Any suggestions to fix that issue on my side so I can test the new file version? |
download the latest version once more. or the whole repo once more. |
GOOD NEWS! Great job, thank you for the super quick fix! DOWNLOAD same video using result_video (nosmooth enabled).zip Suggestion: |
Will do it right away! |
I'm seeing weird artifacts near the lips for some videos even after using |
It’s hard to say without seeing the video. But, yes, high res videos and faces can produce artefacts. |
My first few tries where on some random videos and I noticed that the Mouth not always placed on the correct spot.
So I did few experiments trying to replicate the issue before I post it here.
I've downloaded a Head made in Gan2 (random) and made a simple motion with different speeds as a video, so it's supposed to be easier to track compare to a video with angles and what not since it's just a still image moving around in different speeds so you can SEE the visual issue.
ISSUE Description:
The above are only guess, I only have some experience with other Deep Fake such as:
FaceSwap and DFL so I know that even blurry fast motion could be detected, I just don't know how it works here, probably in a different way but I hope that there is a solution to make even FAST motions trackable to sit on the right position without a lag / delay.
I hope to see this fixed if possible it will be great, please keep up the good work! :)
To keep things clear and simple, the example video is 640x360 - 24fps
Download Video Example Attached as .ZIP file:
result_video.zip
Few Screenshots from the video, shows the mouth not accurate on where it should be:
The text was updated successfully, but these errors were encountered: