PhysioNet/CinC Challenge 2021

Will Two Do? Varying Dimensions in Electrocardiography: The PhysioNet/Computing in Cardiology Challenge 2021

Currently, the 18.04 CUDA docker image is broken (NO_PUBKEY error), causing Docker-CI to fail. See the post in Nvidia's Forum

NOTE that a part of the code for the official phase have been moved to the folder official_phase_legacy.

Updated Version

A benchmark in torch_ecg.

To reproduce the result in this repository, one should stick to torch_ecg @ ed632d1. torch_ecg has been evolving and the latest version has changed a lot.

Digest of Top Solutions (ranked by final challenge score)

1. ISIBrno-AIMT: Custom ResNet + MultiHeadAttention + Custom Loss
2. DSAIL_SNU: SE-ResNet + Custom Loss (from Asymmetric Loss)
3. NIMA: Time-Freq Domain + Custom CNN
4. cardiochallenger: Inception-ResNet + Channel Self-Attention + Custom Loss
5. USST_Med: SE-ResNet + Focal Loss + Data Re-labeling Model
6. CeZIS: ResNet50 + FlowMixup
7. SMS+1: Custom CNN + Hand-crafted Features + Asymmetric Loss
8. DataLA_NUS: EfficientNet + SE-ResNet + Custom Loss

Other teams that are not among official entries, but among unofficial entries:

1. HeartBeats: SE-ResNet + Sign Loss + Model Ensemble

Aizip-ECG-team and Proton had high score on the hidden test set, but did not submitted papers, hence not described here.

Post Challenge Test Results

Test results provided by the challenge organizers. The challenge score on most test sets is comparable to the other teams but is particularly lower on the UMich test set.

Conference Website and Conference Programme

Website, Programme, IEEE Xplore, Poster

Data Preparation

One can download training data from GCP, and use python prepare_dataset -i {data_directory} -v to prepare the data for training

Deep Models

Deep learning models are constructed using torch_ecg, which has already been added as a submodule.

Images

• 2 typical training processes

• "Confusion Matrix" of a typical model on the train-validation set

The "Confusion Matrix" is quoted since it is not really a confusion matrix (the classification is multi-label classification). Its computation can be found here. The diagonal are "true positives", the off-diagonal are "false positives". The "false negatives" are not reflected on this figure. For more matrix plot, ref. Images

Final Results

Final results are on the leaderboard page of the challenge official website or one can find in the offical_results folder.

The last entry failed on the 12-lead UMich test data. It is probably because this last entry used the "no clinical rules" setting, hence normalizations were performed. When the signal has constant value, then dividing by zero (STD) would result in nan values, crashing the whole deep learning pipeline. Actually the last two entries have such constant value records, and normalization is corrected in the data generator. However, corresponding correction was not made in the team_code! To avoid such things, a function for normalizing data in a uniform manner is written.

Citation

@inproceedings{wen_cinc2021,
      title = {{Hybrid Arrhythmia Detection on Varying-Dimensional Electrocardiography: Combining Deep Neural Networks and Clinical Rules}},
     author = {Hao Wen and Jingsu Kang},
  booktitle = {{2021 Computing in Cardiology (CinC)}},
        doi = {10.23919/cinc53138.2021.9662801},
       year = {2021},
      month = {9},
  publisher = {{IEEE}},
}
@article{Kang_2022_cinc2021_iop,
      title = {{A Study on Several Critical Problems on Arrhythmia Detection using Varying-Dimensional Electrocardiography}},
     author = {Jingsu Kang and Hao Wen},
    journal = {{Physiological Measurement}},
        doi = {10.1088/1361-6579/ac6aa3},
       year = {2022},
      month = {6},
  publisher = {{IOP Publishing}},
     volume = {43},
     number = {6},
      pages = {064007}
}

Misc

1. Link to the unsuccessful attemps for CinC2020 of the previous year.
2. bib_lookup for looking up bib entries from DOIs obtained from IEEE Xplore.

CinC2020 | CinC2022 | CinC2023