Skip to content

Official Implementation of 'ReliableSwap: Boosting General Face Swapping Via Reliable Supervision'

Notifications You must be signed in to change notification settings

ygtxr1997/ReliableSwap

Repository files navigation

ReliableSwap: Boosting General Face Swapping Via Reliable Supervision

    Hugging Face Spaces  

Ge Yuan1,2,+, Maomao Li2,+, Yong Zhang2,*, Huicheng Zheng1,* (+ Equal Contributions, * Corresponding Authors)
1 Sun Yat-sen University     2 Tencent AI Lab    

TL;DR: A general face swapping framework that:

🎯 solves no image-level guidance
👩‍❤️‍👩 enhances source identity preservation
♾️ is orthogonal and compatible with existing methods

Fig1

Updates

  • 2023/06/25: Code released!

What Problems We Solve

Fig3

During face swapping training, the re-construction task (used when $X_{\rm{t}}=X_{\rm{s}}$) cannot be used as the proxy anymore when $X_{\rm{t}} \neq X_{\rm{s}}$, lacking pixel-wise supervison $\Gamma$.

How It Works

Fig4

We first use real images $C_{\rm{a}}$ and $C_{\rm{b}}$ to synthesize fake images $C_{\rm{ab}}$ and $C_{\rm{ba}}$. This synthesizing stage preserves the true source identity and target attributes based on Face Reenactment, Multi-Band Blending, and Face Reshaping.

Fig2

Then based on the cycle relationship, for face swapping training stage, we use fake images as inputs while real images as pixel-level supervisons $\Gamma$, keeping the output domain close to the real and natural distribution and solving the non-supervision issue. In this way, the trainable face swapping network is guided to generate source identity-consistency swapping results, while also keeping target attributes.

More details can be found in our project page.

Usage

  1. Environment Preparation
  2. Training
  3. Testing
  4. Constructing Naive/Cycle Triplets by Yourself (Optional)

TODO

  • release code
  • extending to $512^2$ resolution

BibTex

@article{yuan2023reliableswap,
    title={ReliableSwap: Boosting General Face Swapping Via Reliable Supervision},
    author={Yuan, Ge and Li, Maomao and Zhang, Yong and Zheng, Huicheng},
    journal={arXiv preprint arXiv:2306.05356},
    year={2023}
}

📢 Disclaimer

This is not an official product of Tencent. This repository can only be used for personal/research/non-commercial purposes.

Free free to contact us if you feel uncomfortable.

About

Official Implementation of 'ReliableSwap: Boosting General Face Swapping Via Reliable Supervision'

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published