Skip to content
This repository contains the code for the paper `End-to-End Multimodal Emotion Recognition using Deep Neural Networks`.
Branch: master
Clone or download
Latest commit 223cbde Apr 18, 2019
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
LICENSE Create LICENSE Jul 2, 2017
README.md Update README.md Oct 20, 2017
data_generator.py Update data_generator.py Apr 18, 2019
data_provider.py Update data_provider.py Sep 1, 2017
emotion_eval.py Update emotion_eval.py Sep 8, 2017
emotion_train.py Update emotion_train.py Sep 9, 2017
inception_processing.py repository Jun 29, 2017
losses.py Update losses.py Sep 8, 2017
metrics.py Update metrics.py Sep 8, 2017
models.py Update models.py Sep 9, 2017

README.md

End-to-End Multimodal Emotion Recognition using Deep Neural Networks

This package provides training and evaluation code for the end-to-end multimodal emotion recognition paper. If you use this codebase in your experiments please cite:

P. Tzirakis, G. Trigeorgis, M. A. Nicolaou, B. Schuller and S. Zafeiriou, "End-to-End Multimodal Emotion Recognition using Deep Neural Networks," in IEEE Journal of Selected Topics in Signal Processing, vol. PP, no. 99, pp. 1-1. (http://ieeexplore.ieee.org/document/8070966/)

Requirements

Below are listed the required modules to run the code.

  • Python <= 2.7
  • NumPy >= 1.11.1
  • TensorFlow <= 0.12
  • Menpo >= 0.6.2
  • MoviePy >= 0.2.2.11

Content

This repository contains the files:

  • model.py: contains the audio and video networks.
  • emotion_train.py: is in charge of training.
  • emotion_eval.py: is in charge of evaluating.
  • data_provider.py: provides the data.
  • data_generator.py: creates the tfrecords from '.wav' files
  • metrics.py: contains the concordance metric used for evaluation.
  • losses.py: contains the loss function of the training.
  • inception_processing.py: provides functions for visual regularization.

The multimodal model can be downloaded from here : https://www.doc.ic.ac.uk/~pt511/emotion_recognition_model.zip

You can’t perform that action at this time.