Implementation for the paper (submitted to Springer Multimedia Tools and Applications (MTAP) Journal).
Interpretable Multimodal Emotion Recognition using Hybrid Fusion of Speech and Image Data
Puneet Kumar, Sarthak Malik and Balasubramanian Raman
The code files were private till the corresponding research paper's acceptance in Springer MTAP. They will be made publically available soon.
Access to the ‘IIT Roorkee Speech and Image Emotion Recognition (IIT-R SIER) dataset’ can be obtained by through Access Form - IIT-R SIER Dataset.pdf. The dataset is prepared by Puneet Kumar and Sarthak Malik at Machine Intelligence Lab, IIT Roorkee under the supervision of Prof. Balasubramanian Raman. It contains speech utterances, corresponding images and emotion labels (happy, sad, hate, anger).