Realistic Hair Style Try-On: Face and Hair Image Mapping Using Semantic Maps for SDEdit (ECTI-CARD 2022)

Sorayut Meeyim, Phalapat Tektrakul, Pakkaphong Akkabut, Werapon Chiracharit

PROCEEDING ECTI-CARD 2022, Paper, Slide

Abstract

Machine learning-based image generation can create new person face images with new hair colors or hairstyles. This paper presents synthesis and editing method to modify hairstyles in the images by semantic maps. The face and hair images are mapped and inpainted using fast marching method and Stochastic Differential Editing (SDEdit). The experimental results shows that the proposed method controls both hairstyles and color effectively with single target hairstyle image. Moreover, the method is able to generate hairstyles in case of occluded face images.
Keywords: Realistic hair style try-on, Semantic maps, SDEdit

Requirements

• Python 3.8.5 is used. Basic requirements are listed in the 'requirements.txt'.

pip install -r requirements.txt

• Download face segmentation model from this link and put it in image_segmentation/
• Create checkpoints folder and Download checkpoints/celeba_hq.ckpt from this link than put it in checkpoints

Demo app

This is streamlit app that deploy via Google colab. You can find it at this link

Inference

You can run

python inference.py --seg_model_path <image segmentation model> --t <Noise level> --target_image_path <target image path> --source_image_path <source image path>

example:

python inference.py --seg_model_path image_segmentation/face_segment_checkpoints_256.pth.tar --t 500 --target_image_path images/92.jpg --source_image_path images/82.jpg

The results will shown in exp/image_samples folder

Acknowledgement

The structure of this codebase is borrowed from SDEdit.