ChildPredictor

This is the official webpage of the paper "ChildPredictor: A Child Face Prediction Framework with Disentangled Learning", accepted to IEEE TMM, 2022

Arxiv: https://arxiv.org/abs/2204.09962

IEEE Xplore: https://ieeexplore.ieee.org/document/9749880

🚀 🚀 🚀 News:

• May. 19, 2022: We release the data of the FF-Database, please see section 1.1 for the terms of use.

• Apr. 15, 2022: We release the pre-trained models with validation samples for ChildPredictor.

• Mar. 31, 2022: The paper is accepted by the IEEE Transactions on Multimedia.

• Feb. 8, 2022: We release the code for ChildPredictor. We are considerring to release the original data of the collected FF-Database.

1 FF-Database

1.1 Data Download

If you would like to download the FF-Database data, please fill out an agreement to the FF-Database Terms of Use and send it to us at yzzhao2-c@my.cityu.edu.hk.

Please use the institutional email instead of anonymous addresses such as Gmail, QQMail, and hotmail. Make sure your email address is the same as it in the FF-Database Terms of Use.

1.2 Overview

The data collection pipeline is shown as follows:

Some families are shown as follows:

2 Results on Real Families

The generated results on the collected FF-Database:

The generated results on other datasets:

The disentangled learning analysis is as:

The ablation study is as:

3 Implementation

3.1 File Structure

Some files are not included in the current implementation since they are too large. The network architectures can be found in the code folder.

code
│
└───baby_model_pool
│   └───attgan
│   │   │   attgan_without_claloss_baby.pth
│   │   │   attgan_without_ganloss_celeba_baby.pth
│   │   │   attgan_without_ganloss_claloss_celeba_baby.pth
│   │   │   ...
│   └───inverse
│   │   │   Inverse_ProGAN_GAN_ACGAN_start-with-code.pth
│   │   │   Inverse_ProGAN_GAN_MSGAN_ACGAN_start-with-code.pth
│   │   │   Inverse_ProGAN_GAN_MSGAN_ACGAN_start-with-image.pth
│   │   │   ...
│   └───mapping
│   │   └───Mapping_Xencoder_full_ProGAN_GAN_MSGAN_ACGAN_deepArch_multi-gt_v4
│   │   │   │   MappingNet_Batchsize_32_Epoch_298.pth
│   │   └───Mapping_Xencoder_full_ProGAN_GAN_deepArch_multi-gt_v4
│   │   │   │   MappingNet_Batchsize_32_Epoch_298.pth
│   │   └───Mapping_Xencoder_wo-class_ProGAN_GAN_MSGAN_deepArch_multi-gt_v4
│   │   │   │   MappingNet_Batchsize_32_Epoch_298.pth
│   │   │   ...
│   └───ProGAN-ckp
│   │   │   ProGAN_pt_mixtureData_GAN.pth
│   │   │   ProGAN_pt_mixtureData_GAN_ACGAN.pth
│   │   │   ProGAN_pt_mixtureData_GAN_MSGAN.pth
│   │   │   ProGAN_pt_mixtureData_GAN_MSGAN_ACGAN.pth
│   │   │   ...
│
└───babyinverse (Ey)
│   │   ...
|
└───babymapping_1219 (T)
│   │   ...
│
└───Datasets
│   │   ...
│
└───ProGAN (Gy)
│   │   ...
│
└───AttGAN (please refer to AttGAN official webpage)
│   │   ...
│   

3.2 Required Libraries

The implementation is based on CUDA 9.0 and PyTorch 1.1.0. The following packages are needed to be installed:

pytorch==1.1.0
torchvision==0.3.0
tensorflow-gpumkdir ./babymapping_1219/Models/pretrain
mv ./baby_model_pool/ProGAN-ckp/* ./babymapping_1219/Models/pretrain/
tensorboardx
pyyaml
tqdm
easydict

3.3 Testing a Real Face

First, download the pre-trained models from this link. It should be a large zip file with size of approximately 3.9 Gb.

After you have already downloaded the pre-trained models, enter code folder and unzip all the models under the ./code/baby_model_pool folder:

cd code
mkdir baby_model_pool
cd baby_model_pool
unzip Onedrive_baby_model_pool.zip
cd ..

Then, you need to move all the ProGAN pre-trained models under another path:

mkdir ./babymapping_1219/Models/pretrain
mv ./baby_model_pool/ProGAN-ckp/* ./babymapping_1219/Models/pretrain/

Next, you can test some validation samples (we have already put some examples under the code/babymapping_1219 folder):

cd babymapping_1219
python main.py

If you want to change the input images, see lines 38-39 of validation.yaml: https://github.com/zhaoyuzhi/ChildPredictor/blob/main/code/babymapping_1219/yaml/yaml/validation.yaml

3.4 Training

Currently, we do not release the full codes for training due to privacy issue.

3.5 Build Your Own Dataset

Please refer to code_FFDatabase_collection.

4 Network Architectures

5 Some Related Works

• Zaman, Ishtiak and Crandall, David. Genetic-GAN: Synthesizing Images Between Two Domains by Genetic Crossover. European Conference on Computer Vision Workshops, 312--326, 2020.

• Gao, Pengyu and Robinson, Joseph and Zhu, Jiaxuan and Xia, Chao and Shao, MIng and Xia, Siyu. DNA-Net: Age and Gender Aware Kin Face Synthesizer. IEEE International Conference on Multimedia and Expo (ICME), 2021.

• Robinson, Joseph Peter and Khan, Zaid and Yin, Yu and Shao, Ming and Fu, Yun. Families in wild multimedia (FIW MM): A multimodal database for recognizing kinship. IEEE Transactions on Multimedia, 2021.

6 Citation

If you find this work useful for your research, please cite:

@article{zhao2022childpredictor,
  title={ChildPredictor: A Child Face Prediction Framework with Disentangled Learning},
  author={Zhao, Yuzhi and Po, Lai-Man and Wang, Xuehui and Yan, Qiong and Shen, Wei and Zhang, Yujia and Liu, Wei and Wong Chun-Kit and Pang, Chiu-Sing and Ou, Weifeng and Yu, Wing-Yin and Liu, Buhua},
  journal={IEEE Transactions on Multimedia},
  year={2022}
}

Please contact yzzhao2-c@my.cityu.edu.hk for further questions.