This repository is a fork of the original face_de_mask with changes to make it compatible with running on Google Colab.
-
Upload the necessary files to Google Drive
- Download the folder from this link
- Upload the folder to your Google Drive (recommended to put it in
MyDrive/)
-
Run Colab
- Create a fully integrated notebook, including the processing in Deep3DFaceReconstruction-Pytorch and 3DFace repositories necessary to perform face_de_mask
- Fork and modify face_de_mask to work in the Colab environment
- Fork and modify Deep3DFaceReconstruction-Pytorch to work in the Colab environment
The following is the original README
Evaluation code for : Non-Deterministic Face Mask Removal Based On 3D Priors
This paper presents a novel image inpainting framework for face mask removal. Although current methods have demonstrated their impressive ability in recovering damaged face images, they suffer from two main problems: the dependence on manually labeled missing regions and the deterministic result corresponding to each input. The proposed approach tackles these problems by integrating a multi-task 3D face reconstruction module with a face inpainting module. Given a masked face image, the former predicts a 3DMM-based reconstructed face together with a binary occlusion map, providing dense geometrical and textural priors that greatly facilitate the inpainting task of the latter. By gradually controlling the 3D shape parameters, our method generates high-quality dynamic inpainting results with different expressions and mouth movements. Qualitative and quantitative experiments verify the effectiveness of the proposed method.
- Python >= 3.7
- Pytorch >= 1.6
- cv2
- numpy
- PyTorch3D : https://pytorch3d.org/
https://drive.google.com/drive/folders/1-th-qJQGGgzWQF2qrAGr8njJ9EBWHGIR?usp=sharing
- Create BFM related files following the instructions of this project
- Download the pre-trained models and put them in the
ckptsfolder. - Run test.py