Deep Learning has found applications in many predictive tasks relating to more unstructured forms of data over the last few years, such as images, text, audio and video. Many of these tasks seem to be in the vein of a larger direction of predictive modeling that aims to match human-level performance on such tasks, because humans have evolved to specialize in performing intelligent actions on such unstructured data. As a specific branch of AI (also called Affective Computing or Emotion AI) Artificial Emotional Intelligence stands for the study and development of technologies and computers that can read human emotions by means of analyzing body gestures, facial expressions, voice tone, etc. and react appropriately to them.
In the field of human-machine interaction, it has been found that as much as 55% of communication of sentiment takes place through facial expressions and other visual cues. Therefore, training a model to identify facial emotions accurately is an important step towards the development of emotionally intelligent behavior in machines with AI capabilities. Automatic facial expression recognition systems could have many applications, including but not limited to any use case that requires human behavior understanding, detection of mental disorders, and creating a higher quality of virtual assistant for customer-facing businesses. Facial emotion detection is an area of computer that is very critical to the advancement of several aspects of social living. It can be used for:
Human-computer interaction: our engagement with AI-enabled devices can be read and provisioned for (our moods can be read and various actions taken, it can be used to save a human in distress or trouble just by reading their countenance and calling the cops e.g. if someone has been held against their will, fear, anxiety can be read and help can be sought, a patient with a medical condition can be monitored for emergency care etc).
CGI-enabled imaging: it can be used to read faces and project that into the screen to appropriately capture facial movements and expressions. Businesses can also leverage this to proactivaly determine engagement with customers.
Forensic investigation: in homicide cases, it can be used to estimate the last few minutes, seconds, even possibly hours of victims involved
Use Deep Learning and Artificial Intelligence techniques to Create a computer vision model that can accurately detect facial emotions. The model should be able to perform multi-class classification on images of facial expressions, to classify the expressions according to the associated emotion. The key questions:
How do we reliably source (clean) data and if messy, then perform EDA to make it clean? How do we carry out research studies into the application of Artificial Emotional Intelligence and advance its cross-discipline interactions nad applications? How do we achieve an almost perfect performance (precision, recall, F1-Score, etc) that will qialify our work as deployment-ready
- Reliable and accurate identification of facial emotions.
- Gap in human-computer interaction.
The data set consists of 3 folders, i.e., 'test', 'train', and 'validation'. Each of these folders has four subfolders:
‘happy’: Images of people who have happy facial expressions. ‘sad’: Images of people with sad or upset facial expressions. ‘surprise’: Images of people who have shocked or surprised facial expressions. ‘neutral’: Images of people showing no prominent emotion in their facial expression at all.
I have appended images of the training and prediction processes:
- Sample of happy faces
-
Sample of sad faces
-
Sample of neutral faces
- Sample of surprised faces
- Distribution of training dataset
- Distribution of test dataset
- Distribution of validation dataset
- Training and validation accuracy for base model
- Training and validation accuracy for second model
- Training and validation accuracy for vgg16 model
- Training and validation accuracy for resnet model
Training and validation accuracy for efficientnet model
Training and validation accuracy for complex model
- Confusion matrix of final model
- Prediction with final model
- Training and validation accuracy for final model
- Second prediction with final model
- Precision, recall, F1 of final model
