This is a rough outline. A more detailed description will come soon. Please see the publications for the specific models linked in the corresponding sections.
We used virtual reality (VR) to investigate emotional arousal under ecologically valid conditions. 45 subjects experienced virtual roller coaster rides while their neural (EEG) and peripheral physiological (ECG, GSR) responses were recorded. Afterwards, they rated their subject levels of arousal retrospectively on a continuous scale while viewing a recording of their experience.
We tackled the data with three model approaches. The corresponding code can be found in the respective folders. Methods
upload coming soon SPoC Model
Source Power Comodulation (SPoC) decomposes the EEG signal such that it maximizes the covariance between the power-band of the frequency of interest (here alpha, 8-12Hz) and the target variable (ratings).
CSP ModelCommon Spatial Pattern (CSP) algorithm derives a set of spatial filters to project the EEG data onto compontents whose band-power maximally relates to the prevalence of specified classes (here low and high arousal).
This part of the study was published at IEEE VR 2018 in Reutlingen, Germany:
Klotzsche, Mariola, Hofmann, Nikulin, Villringer, & Gaebler. IEEE VR, 2018.
LSTM ModelLong Short-Term Memory (LSTM) recurrent neural networks (RNNs) were trained on alpha-frequency components of the recorded EEG signal to predict subjective reports of arousal (ratings) in a binary (low and high arousal) and a continuous prediction task. The fed EEG components were generated via Spatio Spectral Decomposition (SSD) or SPoC. The SSD emphasizes the frequency of interest (here alpha) while attenuating the adjacent frequency bins. Performances of SPoC-trained models served as benchmark-proxies for models that were trained only on neural alpha information.
Furthermore, we tested whether peripheral physiological responses, here the cardiac information (ECG), increases the performance of the model, and therefore encodes additional information about the subjective experience of arousal.
This part of the study was published at IEEE AIVR 2018 in Taichung, Taiwan:
Hofmann, Klotzsche, Mariola, Nikulin, Villringer, & Gaebler. IEEE AIVR, 2018
Before the main phase of the experiment, we tested the Interceptive Accuracy (IA) of each subject with the MiscellaneousHeart Beat Perception (HBP) task.
CollaboratorsSimon M. Hofmann