Skip to content
No description, website, or topics provided.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
.idea
Demos/VisualEmotionRecognition
KEF
TrainedModels
FERPlus_Vision_FaceChannel_Frame.py
NCD_VisionNetwork_SobelXY.py
OMG_Emotion_Audio_MelSpectrum.py
OMG_Emotion_Crossmodal.py
OMG_Emotion_Evaluation_Audio.py
OMG_Emotion_Evaluation_Cross.py
OMG_Emotion_Evaluation_Face.py
OMG_Emotion_Face.py
RAVDESS_Audio_MelSpectrum_Channel.py
RAVDESS_CrossNetwork_RAVDESS.py
README.md

README.md

This repository holds the models and solutions developed by Pablo Barros based on emotion recognition and learning.

The KEF framework was developed to facilitate the planing and fast prototyping of different scientific experiments. All the examples in this repository use the KEF framework.

Pre-requisites

tensorflow, keras, matplotlib, h5py, opencv-python, librosa, pillow, imgaug, python_speech_features, hyperas

If you want to run on a GPU, install tensorflow-gpu instead of tensorflow

Instructions

Each of the examples here run within the KEF framework. Also, each example needs a specific dataset which is not available here. All the demos and examples here run on Python 2.7.

Hand gesture recognition

  • NCD_VisionNetwork_SobelXY.py: Multichannel Convolution Neural Network for hand posture recognition using the NCD dataset (Barros et al., 2014)

Auditory emotion recognition

  • OMG_Emotion_Audio_MelSpectrum.py: Audio Channel for the OMG-Emotion dataset (Barros et al., 2018)
  • RAVDESS_Audio_MelSpectrum_Channel.py: Audio Channel for the RAVDESS dataset (Barros et al., 2018)

Visual emotion recognition

  • OMG_Emotion_Face.py: Face Channel for the OMG-Emotion dataset (Barros et al., 2018)
  • FERPlus_Vision_FaceChannel.py: Face Channel for the FERPlus dataset (Barros et al., 2018)

Crossmodal emotion recognition

  • OMG_Emotion_Crossmodal.py: Cross Channel for the OMG-Emotion dataset (Barros et al., 2018)
  • RAVDESS_CrossNetwork_MelSpectrum_Channel.py: Cross Channel for the RAVDESS dataset (Barros et al., 2018)

Trained Models

Each of the examples has a pre-trained model associated with it. Please refer to the TrainedModels folder.

Ready-to-Run Demos

Datasets

Follows the links for different corpora that I developed or was involved on the development. Most of the examples here make use of these corpora:

Important references

License

All the examples in this repository are distributed under the Creative Commons CC BY-NC-SA 3.0 DE license. If you use this corpus, you have to agree with the following itens:

  • To cite our associated references in any of your publication that make any use of these examples.
  • To use the corpus for research purpose only.
  • To not provide the corpus to any second parties.

Contact

Pablo Barros - barros@informatik.uni-hamburg.de

You can’t perform that action at this time.