Unsupervised Eyeglasses Removal in the Wild [arXiv]

By Bingwen Hu, Zhedong Zheng, Ping Liu, Wankou Yang and Mingwu Ren. TCYB 2020.

Prerequisites

• Python 3.6, Ubuntu 14.04
• GPU Memory >= 11G
• conda install pytorch>=0.4.1 torchvision cuda91 -y -c pytorch
• conda install -y -c anaconda pip
• conda install -y -c anaconda pyyaml
• pip install tensorboard tensorboardX

Getting started

Clone ERGAN source code

git clone https://github.com/Bingwen-Hu/ERGAN-Pytorch

The folder is structured as follows:

├── ERGAN-Pytorch/
│   ├── configs/                /* Files for configs  
|   |    ├──  celeba.yaml
|   |    ├──  meglass.yaml
│   ├── models/                 /* Files for pretrained model    	
│   ├── outputs/		/* Intermediate image outputs 		
│   ├── datasets/CelebA/
                 ├── trainA/	/* Training set: face images without glasses		
                 ├── trainB/	/* Training set: face images with glasses		
                 ├── testA/	/* Testing set: face images without glasses		
                 └── testB/	/* Testing set: face images with glasses		
│   ├── datasets/MeGlass/
│                ├── trainA/	/* Training set: face images without glasses		
│                ├── trainB/	/* Training set: face images with glasses		
│                ├── testA/	/* Testing set: face images without glasses		
│                └── testB/	/* Testing set: face images with glasses

Dataset Preparation

Download the celebA Dataset Here. Download the MeGlass Dataset Here.

We split the CelebA dataset into one subset with glasses and another without glasses, based on the annotated attributes.

python celeba_glass.py

Note to modify the dataset path to your own path.

Train

Setup the yaml file. Check out configs/celeba.yaml for folder-based dataset organization. Change the data_root field to the path of your downloaded dataset.

python train.py --config configs/celeba.yaml

Intermediate image outputs and model binary files are stored in outputs/celeba.

Test

First, download our pretrained models (Google Drive) for the eyeglasses removal task and put them in models folder.

If you want to test your own data with our pre-trained model, you need to align the data first ( refer to CelebA) or retrain with your own data.

for CelebA:

python test_batch.py --config configs/celeba.yaml --A input_path_A --B input_path_B --output_folder results/celeba --checkpoint models/celeba.pt

for MeGlass:

python test_batch.py --config configs/meglass.yaml --A input_path_A --B input_path_B --output_folder results/meglass --checkpoint models/meglasss.pt

--A The PATH of the test set (without glasses).

--B The PATH of the test set (with glasses).

The results are stored in results/celeba folder and results/meglass folder, respectively.

Citation

If you find ERGAN is useful in your research, please consider citing:

@article{hu2020unsupervised,
  title={Unsupervised eyeglasses removal in the wild},
  author={Hu, Bingwen and Zheng, Zhedong and Liu, Ping and Yang, Wankou and Ren, Mingwu},
  journal={IEEE Transactions on Cybernetics},
  year={2020},
  publisher={IEEE}
}

Related Repos

1. CycleGAN
2. UNIT
3. MUNIT
4. LPIPS
5. FID
6. MeGlass
7. DG-Net

Acknowledgments

Our code is inspired by MUNIT and DG-Net.