-
Notifications
You must be signed in to change notification settings - Fork 493
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
It is difficult to drag the real face #71
Comments
I think the projective image of GAN inversion is pretty good, and the problem you face is the flaw in the DragGAN algorithm. |
Thanks for active reply! Could you give a more intuitive explanation about "the flaw in DragGAN" ? The algorithm works well in the synthetic face from DragGAN but fails in real face after inversion. Is there any difference between latent code (W space) from synthetic face and latent code from real face inversion? |
Well, I don't think this problem is specific for real image. What I mean is that the DragGAN algorithm could fail even if the image is synthetic by GAN without inversion. Besides, the latent code of synthetic face and projected face should not make too much difference with respect to the DragGAN, but the projective one might be less accurate to generate a sharp image. Further, the oscillation might be caused by the similarity of handle points and points around the target positions. So the handle points could not precisely reach the target points. |
Thanks for the profound insights! I agree that inverted real image is obscure and lose a lot of details. However, I came across the situation that I drag the eyebrow up where the similarity of source points and the target points are quite large.
In the optimization of 3rd case, the similarity of I noticed the DragGAN paper uses the PTI for the GAN Inversion. In PTI paper, author claims that the editability of the inverted real image is poor and propose to fine-tune the generator to make inverted image resides in a more editable region. Therefore, can the problem be ameliorated by using PTI for the GAN Inversion? |
Wow, I haven't dive into the GAN inversion that original DragGAN paper uses before, but I think it would be a good idea to try more advanced inversion techniques. |
Hi, thanks for implementing the excellent work!
I try to drag the real face from internet but it seems that DragGAN is difficult to move the handle points to target. The similar operation is easily achieved in synthetic face.
Here is the real face.
![001845](https://private-user-images.githubusercontent.com/129818160/242779676-01437cdf-9cb2-4515-9212-fb4c55955e88.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjA2OTk1MzIsIm5iZiI6MTcyMDY5OTIzMiwicGF0aCI6Ii8xMjk4MTgxNjAvMjQyNzc5Njc2LTAxNDM3Y2RmLTljYjItNDUxNS05MjEyLWZiNGM1NTk1NWU4OC5wbmc_WC1BbXotQWxnb3JpdGhtPUFXUzQtSE1BQy1TSEEyNTYmWC1BbXotQ3JlZGVudGlhbD1BS0lBVkNPRFlMU0E1M1BRSzRaQSUyRjIwMjQwNzExJTJGdXMtZWFzdC0xJTJGczMlMkZhd3M0X3JlcXVlc3QmWC1BbXotRGF0ZT0yMDI0MDcxMVQxMjAwMzJaJlgtQW16LUV4cGlyZXM9MzAwJlgtQW16LVNpZ25hdHVyZT0wNzMyNjVjMzhiNzg4NzJiNjZjOGViODY1ZDcwNjczNjJlZDczMzY5MWQ2YjA2ZjFmOGFhMzA4ZThlOTBjMmEwJlgtQW16LVNpZ25lZEhlYWRlcnM9aG9zdCZhY3Rvcl9pZD0wJmtleV9pZD0wJnJlcG9faWQ9MCJ9.q_l_vtjQFswcsQeDFpaDCiEqoj196pSbuoEfdlFGHeY)
Here is the process of the drag. The handle points only oscillate around the starting position. The algorithm takes 50 steps. More optimization steps do not make any help.
https://github.com/Zeqiang-Lai/DragGAN/assets/129818160/9f566779-a6ea-4528-a8bf-1ae775f0b492
Is there problem in GAN Inversion? Or this is an inherent flaw in the DragGAN algorithm?
The text was updated successfully, but these errors were encountered: