TM-HGNN

[ACL 2023 Oral] Clinical Note Owns its Hierarchy: Multi-Level Hypergraph Neural Networks for Patient-Level Representation Learning [Paper]

Leveraging knowledge from electronic health records (EHRs) to predict a patient{'}s condition is essential to the effective delivery of appropriate care. Clinical notes of patient EHRs contain valuable information from healthcare professionals, but have been underused due to their difficult contents and complex hierarchies. Recently, hypergraph-based methods have been proposed for document classifications. Directly adopting existing hypergraph methods on clinical notes cannot sufficiently utilize the hierarchy information of the patient, which can degrade clinical semantic information by (1) frequent neutral words and (2) hierarchies with imbalanced distribution. Thus, we propose a taxonomy-aware multi-level hypergraph neural network (TM-HGNN), where multi-level hypergraphs assemble useful neutral words with rare keywords via note and taxonomy level hyperedges to retain the clinical semantic information. The constructed patient hypergraphs are fed into hierarchical message passing layers for learning more balanced multi-level knowledge at the note and taxonomy levels. We validate the effectiveness of TM-HGNN by conducting extensive experiments with MIMIC-III dataset on benchmark in-hospital-mortality prediction.

Requirements

• CUDA=11.3
• cuDNN=8.2.0
• python=3.9.12
• pandas=1.4.2
• torch=1.11.0
• torch_geometric=2.1.0

Usage

We follow MIMIC-III Benchmark (Harutyunyan et al.) for preprocess clinical notes. The preprocessed NOTEEVENTS data for in-hospital-mortality should be in data/DATA_RAW/in-hospital-mortality, divided into two folders (train_note and test_note).

Prepare Notes

python graph_construction/prepare_notes/extract_cleaned_notes.py
python graph_construction/prepare_notes/create_hyper_df.py

extract_cleaned_notes.py cleans clinical notes in data/DATA_RAW/in-hospital-mortality, which results in column "Fixed TEXT" in each csv file. Word2vec token embeddings with 100 dimensions are created and saved in data/DATA_RAW/root/word2vec_100.

create_hyper_df.py creates dataframe from data/DATA_RAW/in-hospital-mortality where each row represents each word. The results are stored in data/DATA_PRE/in-hospital-mortality, divided into two folders (train_hyper and test_hyper).

Construct Multi-level Hypergraphs

python graph_construction/prepare_notes/PygNotesGraphDataset.py --split train
python graph_construction/prepare_notes/PygNotesGraphDataset.py --split test

PygNotesGraphDataset.py creates multi-level hypergraphs with cutoff in data/IMDB_HCUT/in-hospital-mortality.

Model Train

python tmhgnn/train.py

Citation

@inproceedings{kim-etal-2023-clinical,
    title = "Clinical Note Owns its Hierarchy: Multi-Level Hypergraph Neural Networks for Patient-Level Representation Learning",
    author = "Kim, Nayeon  and  Piao, Yinhua  and  Kim, Sun",
    booktitle = "Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)",
    year = "2023",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2023.acl-long.305",
    pages = "5559--5573",
}