Detecting Cartoon Characters’s Emotions Using Transfer Learning

Feifei Wang, Yutong Zhang

Goal

Emotion Classification of Cartoon Characters of different style (anime vs. 3D cartoon)

Solution

• Transfer learning and fine-tuning
• Compare the accuracy of different pretrained models & baseline CNNs

Values

• Few studies explored this subject before => improve image search result quality
• Understand how neural networks differentiate emotions of fictional figures, whose characteristics vary dramatically between artists

Challenges

• Animated faces have different characteristics from real human faces
• The cartoon facial emotion datasets are limited, with small sizes that is susceptible to overfitting

Data Sets

1. Facial Expression Research Group 2D Database (FERG-DB)

55767 annotated face images of 6 characters

{'angry': 0,
  'crying': 1,
  'embarrassed': 2,
  'happy': 3,
  'pleased': 4,
  'sad': 5,
  'shock': 6 }

2. Manga Facial Expressions Data Set (462 images)

{'anger': 0,
 'disgust': 1,
 'fear': 2,
 'joy': 3,
 'neutral': 4,
 'sadness': 5,
 'surprise': 6 }

Code Structure

all in .ipynb, separated by models and datasets

• trained-from-scratch CNN （作为baseline model，其他的accuracy可以和它compare）
• GoogleNet
• ResNet50

Result

GoogleNet

• Use ’val_categorical_accuracy’ to evaluate accuracy
• Overall top 3 performance:
  • L2 Regularization
  • baseline + Batch Norm + 2 Dense 64 Layer
  • baseline
• When the dataset is small
  • Changing the structure of the model is able to increase the accuracy and control overfitting, with mild effect on runtime
• When the dataset is large:
  • GoogleNet is faster the baseline

ResNet50 and Vanilla CNNs