Code for "Unsupervised Discovery of Object Landmarks as Structural Representations"
Project page: http://www.ytzhang.net/projects/lmdis-rep/
The code works with TensorFlow 1.7 (but not higher than 1.7) and Python 3.6 (or higher).
Please email Yuting Zhang if any of the download links are broken.
Yuting Zhang, Yijie Guo, Yixin Jin, Yijun Luo, Zhiyuan He, Honglak Lee, "Unsupervised Discovery of Object Landmarks as Structural Representations," IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2018. [arxiv]
What are included
For experiment on CelebA and AFLW dataset
- Demo using pretrained models (detection and visualization on individual images)
- Training code
- Evaluation code
Data to download
- The CelebA dataset, saved in the
- ground truth landmark annotation for pre-processed CelebA images, saved in the
- The AFLW dataset of preprocessed images, so that the face is on the similar position as face in CelebA images, saved in
- ground truth landmark annotation for pre-processed AFLW images, saved in
- The CAT dataset of preprocessed images, so that the face is on the similar position as face in CAT images, saved in
- ground truth landmark annotation for pre-processed CAT images, saved in
Pretrained Models to download
The pretrained models for CelebA dataset can be obtained via this link, which detect 10 or 30 landmarks on the image.
The pretrained models for AFLW dataset can be obtained via this link, which detect 10 or 30 landmarks on the image.
The pretrained models for AFLW dataset can be obtained via this link, which detect 10 or 20 landmarks on the image.
./download_cat.sh will automatically download pretrained models and data for experiment on each dataset. The pretrained model will be saved in
pretrained_results/cat_20 . And the data will be saved in
data/cat_images. Note that you should download the CelebA data by yourself into
Demo on CelebA image samples using pre-trained model (quick demo)
You can run a quick demo on CelebA images.
Download our pretrained model on CelebA.
Put the pretrained model in the same directory as defined in
one_step_test_celeba_demo.sh, the directory is
After that, run
You should be able to view the visualization of the landmark discovery results in the
demo/outputfolder created under the root of the project. If
pretrained_results/celeba_10, there are 10 detected landmarked on each image. If
pretrained_results/celeba_30, there are 30 detected landmarked on each image.
To perform detection on other human face images, you can just put the images you are interested in into the
demo/input folder, and rerun
./one_step_test_celeba_demo.sh to see the detected landmarks on these images in
- Train the model on CelebA dataset for 10 landmarks:
- Train the model on CelebA dataset for 30 landmarks:
- Train the model on AFLW dataset for 10 landmarks:
python exp-ae-aflw-10.py, the AFLW is finetuned based on pretrained model for CelebA dataset, so we must have
- Train the model on AFLW dataset for 30 landmarks:
python exp-ae-aflw-30.py, the AFLW is finetuned based on pretrained model for CelebA dataset, so we must have
- Train the model on CAT dataset for 10 landmarks:
- Train the model on CAT dataset for 30 landmarks:
- Test the model on CelebA dataset
- Test the model on AFLW dataset
- Test the model on CAT dataset
one_step_test_cat.sh, you can specify
SPECIFIC_MODEL_DIR as the path of folder saving the trained checkpoint, and
SNAPSHOT_ITER as the number of snapshot step you would like to test. If the snaphot step is not specified, the script automatically test on the lastest checkpoint.
vis folder, call the matlab function
vppAutoKeypointImageRecon(result_path, step, sample_ids, save_to_file, type_ids)
result_path is the folder saving training results,
step is the training step you want to test on (the step must be saved in corresponding
samples_ids is the id of test samples you are interested in,
save_to_file is whether you would like to save the visualization figures,
type_ids is 'data-encoded' or 'recon-encoded'.
vppAutoKeypointImageRecon('../results/celeba_10/', 220000, 1:20, false, 'data-encoded') will visualize the discovered keypoints on test images 1~20 in test dataset.
- The landmarks outputs by our model are in the ordering of
yx(not the usual
- Models on other datasets and more code updates are coming soon.