For image restoration, methods leveraging priors from generative models have been proposed and demonstrated a promising capacity to robustly restore photorealistic and high-quality results. However, these methods are susceptible to semantic ambiguity, particularly with images that have obviously correct semantics such as facial images. In this paper, we propose a semantic-aware latent space exploration method for image restoration (SAIR). By explicitly modeling semantics information from a given reference image, SAIR is able to reliably restore severely degraded images not only to high-resolution and highly realistic looks but also to correct semantics. Quantitative and qualitative experiments collectively demonstrate the superior performance of the proposed SAIR.
This is the code for the paper "Self-Supervised Face Image Restoration with One-shot Reference" (SAIR)
To avoid the torturous environment configuration, we recommend you use Anaconda to finish the next configuration.
conda create -n SAIR python=3.6source activate SAIR && conda install cudatoolkit=10.1pip3 install -r requirement.txt -f https://download.pytorch.org/whl/torch_stable.html- Download "stylegan2-ffhq-config-f.pt" by Google Drvier and "irestnet50.pth" by Google Driver. Put these two model files in the directory "pretrained_models/"
- Put the guide image and the corresponding inverse latent code in "guide_info/your_dir/". We solve for the inverse latent code by e4e . In "guide_info", we provide two examples.
- All done.
python run.py -i test_img/obama.png -gl guide_info/obama/latents.pt -gi guide_info/obama/ref.png -e disgust -ee -eh
@article{guo2023selfsupervised,
title={Self-Supervised Face Image Restoration with a One-Shot Reference},
author={Guo, Yanhui and Luo, Fangzhou and Xu, Shaoyuan},
journal={The 49th IEEE International Conference on Acoustics, Speech, & Signal Processing (ICASSP)},
year={2024}
}