Fine-Grained Lip Image Segmentation using Fuzzy Logic and Graph Reasoning

Introduction | Preparation | Inference | License

Authors

Lei Yang, Shilin Wang, Alan Wee-Chung Liew

Introduction

This is the official implementation of Fine-Grained Lip Image Segmentation using Fuzzy Logic and Graph Reasoning.

Using this repository, you can get fine-grained segmentation results of lip image in natural scenes.

Demo

Preparation

1. Clone the repository.

   git clone https://github.com/YangLeiSX/FLRSeg.git
   cd FLRSeg

2. Setup the environment.

   conda env create -f exvironments.yaml

3. Download the pre-trained model from here(key: bj38).

Inference

1. (optional) Crop lip images from face images.

   python crop_mouth.py
       --input face.jpg
       --output mouth.jpg

2. Inference lip images.

   python predict.py 
       --model weights/params-b5b19c.pth 
       --input mouth.jpg
       --multiscale
       --cuda

Dataset

Fine-grained Lip Region Segmentation dataset.

Facial images are selected from Visual Speaker Authentication (VSA) dataset. Lip region are localized and cropped to the center of images.

Five semantics categories are annotated. Annotation conclude background(facial pixels), lip, teeth, tongue, inner cavity.

Please contact me with academic/institutional email for the full dataset.

Citation

If you use the FLRSeg models and/or dataset, please consider citing the following paper:

@article{yang2023fine,
  title={Fine-Grained Lip Image Segmentation using Fuzzy Logic and Graph Reasoning},
  author={Yang, Lei and Wang, Shilin and Liew, Alan Wee-Chung},
  journal={IEEE Transactions on Fuzzy Systems},
  year={2023},
  publisher={IEEE},
  url={https://doi.org/10.1109/TFUZZ.2023.3298323},
  doi={10.1109/TFUZZ.2023.3298323},
}

License

It is noted that the code can only be used for comparative or benchmarking purposes. Users can only use code supplied under a License for non-commercial purposes.

Contact

Lei Yang