ArtFacePoints: High-resolution Facial Landmark Detection in Paintings and Prints

This is the source code for our conference article "ArtFacePoints: High-resolution Facial Landmark Detection in Paintings and Prints" accepted at the VISART VI Workshop at the ECCV 2022.

The source code and pre-trained models are only for research purposes. If you use the code, please cite our paper

  @InProceedings{Sindel2022ArtFacePoints,
	title={{ArtFacePoints: High-resolution Facial Landmark Detection in Paintings and Prints}},
	author={Aline Sindel and Andreas Maier and Vincent Christlein},
	year={2023},
	booktitle={Computer Vision – ECCV 2022 Workshops. ECCV 2022},
	doi={10.1007/978-3-031-25056-9_20},
	pages={298-313}
  }

Requirements

Install the requirements using pip or conda (python 3):

• torch (we used PyTorch 1.10. Cuda 11.3)
• torchvision
• opencv-contrib-python-headless
• numpy

Usage

We provide the pre-trained model and the inference code to test ArtFacePoints for artwork images of size 1024 x 1024 which are cropped to the facial region.

Pre-trained weights

The pre-trained weights of ArtFacePoints from the paper can be downloaded here.

Move them into the folder: "/weights"

ArtFacePoints Inference:

Run detect_facial_landmarks_ArtFacePoints.py

Some settings have to be specified, as described in the python file, such as the dataset and output folders and the model path.

Landmark data format

The facial landmark predictions are saved in the .pts format (as used by menpo): 68 2d coordinates (x,y) with Matlab indexing (1,1024), because we use menpo for online geometric data augmentation during training.

Acknowledgments

Code is partly based on:

• Encoder-decoder architecture based on the generator network of CycleGAN-Pix2Pix
• Soft-argmax
• Geometric style augmentation for training adopted from Face of Art
• Artistic style transfer AdaIN using AdaIN-pytorch and image-to-image translation using CycleGAN to generate synthetic training data

@author Aline Sindel