Team: Guangyi Zhang and Ali Etemad
parent_dir/
│
├── EPiC_code/ # source code directory
│ ├── ecg_net.py # end-to-end deep learning modules: Conv1D and Transformer
│ ├── parsing.py # command line arguments (e.g., learning rate, choice of signal modalities, scenario numbers, fold numbers)
│ ├── preprocessing.py # preprocessing for ECG and GSR
│ ├── pretraining_final.py # pretraining using entire training data from the same fold under the same scenario
│ ├── signal_main.py # filtering data and segmenting data with sliding windows
│ ├── test_final.py # generation of test results with two options: 1. training from scratch and 2. retraining using the pre-trained model weights
│ ├── train.py # obtaining validation results with two options: 1. training from scratch and 2. retraining using the pre-trained model weights
│ ├── train_test_slit.py # train/test split
│ ├── train_val_slit.py # train/val split with the split startegy according to train/test split one, random seed fixed
│ └── utils.py # containing frequently used helper functions
│
├── bash scripts/ # comands to generate val/test results for each scenario
│ ├── EPiC_s1.sh # scenario 1: experiment on test set
│ ├── EPiC_s2.sh # scenario 2 (5 folds): experiment on test set
│ ├── EPiC_s3.sh # scenario 3 (4 folds): experiment on test set
│ ├── EPiC_s4.sh # scenario 4 (2 folds): experiment on test set
│ ├── EPiC_v1.sh # scenario 1: experiment on validation set
│ ├── EPiC_v2.sh # scenario 2 (5 folds): experiment on validation set
│ ├── EPiC_v3.sh # scenario 3 (4 folds): experiment on validation set
│ ├── EPiC_v4.sh # scenario 4 (3 folds): experiment on validation set
│ └── submission.sh # move path to the one required by the organizers
│
├── submissions/ # results
│ └── results.zip # containing results in .csv format for all scenarios
│
├── README.md # documentation file with project information
│
├── requirements.txt # required libs and their versions.
-
Data Preprocessing: This step entails cleansing and preparing the data for analysis, which includes filtering, normalization, and segmentation of physiological signals such as ECG and GSR. We opt for a window length of 10 seconds.
-
Model Selection: Next, we select end-to-end deep learning-based regression models (Convolutional Neural Network and Transformer) to predict valence and arousal ratings based on the acquired ECG representations.
-
Train/Validation Split: We adhere to the train-test split protocols provided by the organizer, splitting the validation set from the training set in each scenario.
-
Training strategies: 1) training from scratch, and 2) pretraining the model on the entire training data from the same fold in the same scenario, followed by retraining the model.
-
Evaluation: In scenario 1, we conduct subject-dependent and video session-dependent experiments. In each fold of scenario 2, we carry out session-dependent experiments. In each fold of both scenarios 3 and 4, we perform subject-dependent experiments.
-
Final results: We assess several baselines on the validation set and select the best one (according to averaged validation RMSE values) for the test set to obtain the final results for submission.
The usage of this code is straightforward. The only modification needed is to change the '/path/to/folder' in 'utils.py' and any bash files you wish to use. You can opt to acquire validation results or generate test annotations with the bash files provided in the 'bash_scripts' folder. Finally, if you want to generate test annotations in the format required by the organizers, please refer to the example in 'submission.sh'.
| Model/Train | Scratch (ECG Only) | Retrain (ECG Only) |
| ------------- | ---------------------- | ---------------------- |
| 1-D CNN | N/A | Attempt 1 (RMSE: 1.29) |
| Transformer | Attempt 2 (RMSE: 1.22) | Attempt 3 (RMSE: 1.31) |
After evaluating the root mean square error (RMSE) values on a portion of the test data, as presented in the table, we have selected 'Attempt 2' as our final submission.
To replicate the results of the final submission, execute the bash files "EPiC_s1", "EPiC_s2", "EPiC_s3", and "EPiC_s4", while commenting out commands containing 'retrain' that were used for "Attempt 3".
If you wish to reproduce the results of 'Attempt 1', modify the module in 'ecg_net.py' by changing 'Transformer' to 'Conv-1D' and run the aforementioned bash files, making sure to comment out commands with 'scratch'.
Refer to the 'Usage' section above for instructions on how to move results from saved folders (e.g., './ecg/scratch' or './ecg/retrain') into the format required by the EPiC organizers.
Please note that this project was developed under specific constraints which may have affected the performance and generalization of the model. Due to the current intense research at the University of Toronto, I only have the chance to dedicate two entire weekends with 40 hours in total to form this version of code or solution. Additionally, I faced the challenge of limited GPU resources during the development process.
Despite these constraints, I have built a complete pipeline that can be utilized by other researchers to build upon and potentially improve the model's performance. We encourage the community to explore and experiment with this pipeline to optimize the results further.
In this project, we focused on single channel ECG out of a total of 8 available signals including other modalities. However, we believe that incorporating additional modalities could lead to better results. Our code is designed to be easily extendable, and we welcome researchers to modify and adapt it to their specific needs.
We appreciate any contributions, suggestions, or improvements that others might offer to enhance the performance and applicability of our work. Please feel free to fork this repository, submit pull requests, or open issues to discuss potential improvements.
Should you have any questions, please feel free to contact me at gzhang14@mgh.harvard.edu
This work is licensed under a Creative Commons Attribution 4.0 International License.
