-
Notifications
You must be signed in to change notification settings - Fork 0
/
papers.yaml
128 lines (114 loc) · 11 KB
/
papers.yaml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
- title: "Noise-Based Augmentation Techniques for Emotion Datasets: What Do We Recommend?"
date: 2020-07-01
categories: [Data Augmentation, Emotion Recognition, Speech and Audio, Empirical Analysis]
exthref: https://arxiv.org/abs/2104.08806
description: Multiple noise-based data augmentation approaches have been proposed to counteract this challenge in other speech domains. But, unlike speech recognition and speaker verification, the underlying label of emotion data may change given the addition of noise. In this work, we propose a set of recommendations for noise-based augmentation of emotion datasets based on human and machine performance evaluation of generated realistic noisy samples using multiple categories of environmental and synthetic noise.
authors: Mimansa Jaiswal, Emily Mower Provost
publisher: ACL-SRW
display-date: 2020
talk: https://slideslive.com/38928670/noisebased-augmentation-techniques-for-emotion-datasets-what-do-we-recommend
- title: "MuSE: Multimodal Stressed Emotion Dataset"
date: 2020-05-01
categories: [Data Collection, Confounding Factors, Emotion Recognition, Speech and Audio]
exthref: http://www.lrec-conf.org/proceedings/lrec2020/pdf/2020.lrec-1.187.pdf
description: This paper presents a dataset, Multimodal Stressed Emotion (MuSE), to study the multimodal interplay between the presence of stress and expressions of affect. We describe the data collection protocol, the possible areas of use, and the annotations for the emotional content of the recordings.
authors: Mimansa Jaiswal, Cristian-Paul Bara, Yuanhang Luo, Rada Mihalcea, Mihai Burzo, Emily Mower Provost
publisher: LREC
display-date: May 2020
- title: "Privacy Enhanced Multimodal Neural Representations for Emotion Recognition"
date: 2020-02-01
categories: [Confounding Factors, Emotion Recognition, Speech and Audio, Text, Model Training, ]
exthref: https://arxiv.org/pdf/1910.13212.pdf
description: This paper presents a dataset, Multimodal Stressed Emotion (MuSE), to study the multimodal interplay between the presence of stress and expressions of affect. We describe the data collection protocol, the possible areas of use, and the annotations for the emotional content of the recordings.
authors: Mimansa Jaiswal, Emily Mower Provost
publisher: AAAI and NeuRIPS-W
display-date: Feb 2020
- title: "Human-Centered Metric Design to Promote Generalizable and Debiased Emotion Recognition"
date: 2022-10-01
categories: [Debiasing, Emotion Recognition, Text, Model Training, Empirical Analysis, Generalization, Evaluation, Metric Design, Interpretation]
exthref: https://arxiv.org/abs/2104.08792
description: Metrics for emotion recognition can be challenging due to their dependence on subjective human perception. This paper proposes a template formulation that derives human-centered, automatic, optimizable evaluation metrics for emotion recognition models. The template uses model explanations and sociolinguistic wordlists and can be applied to a sample or whole dataset. The proposed metrics include generalizability and debiasing improvement, and are tested on three models, datasets and sensitive variables. The metrics correlate with the models' performance and biased representations, and can be used to train models with increased generalizability, decreased bias, or both. The template is the first to provide quantifiable metrics for training and evaluating generalizability and bias in emotion recognition models.
authors: Mimansa Jaiswal, Emily Mower Provost
publisher: arXiv
display-date: Nov 2022
- title: "Identifying Mood Episodes Using Dialogue Features from Clinical Interviews"
date: 2019-09-01
categories: [Emotion Recognition, Text, Model Training, Speech and Audio, Empirical Analysis, Mental Health, Dialogue]
exthref: https://arxiv.org/pdf/1910.05115.pdf
description: Mental health professionals assess symptom severity through semi-structured clinical interviews. During these interviews, they observe their patients’ spoken behaviors, including both what the patients say and how they say it. In this work, we move beyond acoustic and lexical information, investigating how higher-level interactive patterns also change during mood episodes.
authors: Zakaria Aldeneh, Mimansa Jaiswal, Emily Mower Provost
publisher: Interspeech
display-date: Sep 2019
- title: "MuSE-ing on the Impact of Utterance Ordering on Crowdsourced Emotion Annotations"
date: 2019-05-01
categories: [Emotion Recognition, Data Annotation, Empirical Analysis, Crowdsourcing]
exthref: https://arxiv.org/pdf/1903.11672.pdf
description: Emotion expression and perception are inherently subjective. There is generally not a single annotation that can be unambiguously declared “correct.” As a result, annotations are colored by the manner in which they were collected, i.e., with or without context.
authors: Mimansa Jaiswal, Zakaria Aldeneh, Cristian-Paul Bara, Yuanhang Luo, Mihai Burzo, Rada Mihalcea, Emily Mower Provost
publisher: ICASSP
display-date: May 2019
- title: "The PRIORI Emotion Dataset: Linking Mood to Emotion Detected In-the-Wild"
date: 2018-09-01
categories: [Emotion Recognition, Model Training, Speech and Audio, Empirical Analysis, Mental Health]
exthref: https://arxiv.org/pdf/1806.10658.pdf
description: This paper presents critical steps in developing this pipeline, including (a) a new in the wild emotion dataset, the PRIORI Emotion Dataset, (b) activation/valence emotion recognition baselines, and, (c) establish emotion as a meta-feature for mood state monitoring.
authors: Soheil Khorram, Mimansa Jaiswal, John Gideon, Melvin McInnis, Emily Mower Provost
publisher: Interspeech
display-date: Sep 2018
- title: "'Hang in there': Lexical and Visual Analysis to Identify Posts Warranting Empathetic Responses"
date: 2017-12-01
categories: [Emotion Recognition, Mental Health, Text]
exthref: https://arxiv.org/pdf/1903.05210.pdf
description: Saying "You deserved it!" to "I failed the test" is not a good idea. In this paper, we propose a method supported by hand-crafted features to judge if the discourse or statement requires an empathetic response.
authors: Mimansa Jaiswal, Sairam Tabibu, Erik Cambria
publisher: FLAIRS
display-date: Dec 2017
- title: "'The Truth and Nothing But The Truth': Multimodal Analysis for Deception Detection"
date: 2016-07-01
categories: [Emotion Recognition, Mental Health, Multimodal, Text, Speech and Audio]
exthref: https://arxiv.org/pdf/1903.04484.pdf
description: We propose a data-driven method (SVMs) for automatic deception detection in real-life trial data using visual (OpenFace) and verbal cues (Bag of Words).
authors: Mimansa Jaiswal, Sairam Tabibu, Rajiv Bajpai
publisher: ICDM-W
display-date: Jul 2017
- title: "Capturing Mismatch between Textual and Acoustic Emotion Expressions for Mood Identification in Bipolar Disorder"
date: 2023-09-01
categories: [Emotion Recognition, Mental Health, Text, Speech and Audio, Metric Design]
description: Emotion is expressed through language, vocal and facial expressions. Lack of emotional alignment between modalities is a symptom of mental disorders. We propose to quantify the mismatch between emotion expressed through language and acoustics, which we refer to as Emotional Mismatch (EMM). EMM patterns differ between symptomatic and euthymic moods. EMM statistics serve as an effective feature for mood recognition, reducing annotation cost while preserving mood identification.
authors: Minxue Niu, Amrit Romana, Mimansa Jaiswal, Melvin McInnis, Emily Mower Provost
publisher: Interspeech
display-date: Sep 2023
- title: "Mind the Gap: On the Value of Silence Representations to Lexical-Based Speech Emotion Recognition"
date: 2022-09-01
categories: [Emotion Recognition, Text, Speech and Audio, Model Training, Interpretation]
description: Silence is crucial in speech perception, conveying emphasis and emotion. However, little research has been done on the effect of silence on linguistics and emotion recognition. We present a novel framework that fuses linguistic and silence representations for emotion recognition in naturalistic speech. Two methods to represent silence are investigated, with results showing improved performance. Modeling silence as a token in a transformer language model significantly improves performance on the MSP-Podcast dataset. Analyses show that silence emphasizes the attention of its surrounding words.
authors: Matthew Perez, Mimansa Jaiswal, Minxue Niu, Cristina Gorrostieta, Matthew Roddy, Kye Taylor, Reza Lotfian, John Kane, Emily Mower Provost
publisher: Interspeech
display-date: Sep 2022
exthref: https://www.isca-speech.org/archive/pdfs/interspeech_2022/perez22_interspeech.pdf
- title: "Designing Interfaces for Delivering and Obtaining Generation Explanation Annotations"
date: 2023-03-01
categories: [Text, Data Annotation, Design]
description: Designing a user interface where human annotators can provide explanations for text data. This can help improve the transparency and interpretability of machine learning models, as well as improve their performance.
authors: Mimansa Jaiswal
publisher: Submission
display-date: Mar 2023
demo: https://human-in-loop-explanation-annotation.vercel.app/
repo: https://github.com/mimansajaiswal/hil-text-annotation
relatednote: ../notes/explanation_annotation_interfaces.html
- title: "Controlled Evaluation of Explanations: What Might Have Influenced Your Model Explanation Efficacy Evaluation?"
date: 2022-03-01
categories: [Text, Evaluation, Metric Design, Schema, Interpretation, Data Annotation]
description: Factors affecting explanation efficacy include the algorithm used and the end user. NLP papers focus on algorithms for generating explanations, but overlook other factors. This paper examines how saliency-based explanation methods for machine learning models change with controlled variables. We aim to provide a standardized list of variables to evaluate these explanations and show how SoTA algorithms can have different rankings when controlling for evaluation criteria.
authors: Mimansa Jaiswal, Minxue Niu
publisher: Submission
display-date: Mar 2022
relatednote: ../notes/controlled_evaluation_of_explanations.html
- title: "CAPSTONE: Capability Assessment Protocol for Systematic Testing of Natural Language Models Expertise"
date: 2023-03-01
categories: [Text, Evaluation, Metric Design, Schema, Interpretation, Data Annotation, Foundation Models]
description: Prompt-based language models introduce uncertainty to classification and require users to try multiple prompts with varying temperatures to find the best fit. However, this approach lacks the ability to capture implicit differences in prompts and provide adequate vocabulary. To address this, a text annotation framework is proposed to provide a structured approach to prompt definition and annotation. Better validation structures and structured prompts are necessary for using prompt-based systems at scale for labeling or retrieval.
authors: Mimansa Jaiswal
publisher: Submission
display-date: Mar 2023
relatednote: ../notes/capstone.html