KoIn_Benchmark_Dataset - Korean Influencer

New Benchmarks for Asian Facial Recognition Tasks: Face Classification with Large Foundation Models

• This repository provides Korean Influencer(KoIN) Dataset and PyTorch implementations for Image Classification models.

Authors

Jinwoo Seo, Soora Choi, Eungyeom Ha, BeomJune Kim, Dongbin Na

Abstract

The face classification system is an important tool for recognizing personal identity properly. This paper introduces a new Large-Scale Korean Influencer Dataset named KoIn. Our presented dataset contains many real-world photos of Korean celebrities in various environments that might contain stage lighting, backup dancers, and background objects. These various images can be useful for training classification models classifying K-influencers. Most of the images in our proposed dataset have been collected from social network services (SNS) such as Instagram. Our dataset, KoIn, contains over 100,000 K-influencer photos from over 100 Korean celebrity classes. Moreover, our dataset provides additional \textit{hard case} samples such as images including human faces with masks and hats. We note that the hard case samples are instrumental in evaluating the robustness of the classification systems. We have extensively conducted several experiments utilizing various classification models to validate the effectiveness of our proposed dataset. Specifically, we demonstrate that recent state-of-the-art (SOTA) foundation architectures show decent classification performance when trained on our proposed dataset. In this paper, we also analyze the robustness performance against hard case samples of large-scale foundation models when we fine-tune the foundation models on the normal cases of the proposed dataset, KoIn. Our presented dataset and codes will be publicly available for academic purposes after the paper is accepted.

Datasets

• The dataset is divided into three distinct groups based on the difficulty of Korean influencer / Asian image classification, the Normal cases, the Hard cases, and the Group cases.

  

• This repository provides (1) Normal case training dataset, (2) Normal case test dataset, (3) Hard case test dataset, (4) Group case test dataset

  Dataset/
        normal_cases/
            train/
                0000/
                    0.jpg
                    .....
                0001/
                ....
                0099/
            test/
        hard_cases/
            test/
        Group_cases/
            test/

• The normal case training dataset contains 100,000 images.

• The normal case test dataset contains 5,000 images.

• The hard case test dataset contains 1,000 images.

• The group case test dataset contains 500 images.

Source Codes

• Datasets and source codes that match the experimental environment.

  	Dataset
  KoIn10	KoIn10
  KoIn50	KoIn50
  KoIn100	KoIn100

  	Architecture
  ResNet	ResNet_foundation_model
  DenseNet	DenseNet_foundation_model
  EfficientNet	EfficientNet_foundation_model
  BiT-M	BiT-m_foundation_model
  CLIP	CLIP_foundation_model

Experiment result

• Experimental results with KoIn 50 dataset. We experimented with CNN architectures ResNet, DenseNet, EfficientNet, and few-shot learning architectures BiT, CLIP.

  

• KoIn hard / group case performance experiment. We selected a model that performed well in the normal case.

  

Citation

If this work can be useful for your research, please cite our paper:

@article{seo2023new,
  title={New Benchmarks for Asian Facial Recognition Tasks: Face Classification with Large Foundation Models},
  author={Seo, Jinwoo and Choi, Soora and Ha, Eungyeom and Kim, Beomjune and Na, Dongbin},
  journal={arXiv preprint arXiv:2310.09756},
  year={2023}
}