This repository contains code for the ICASSP'26 paper: GraphPL: Leveraging GNN for Efficient and Robust Modalities Imputation in Patchwork Learning. Our repository is built on CLAP.
- mmnist: dataset, networks and running files for the PolyMNIST dataset.
- mnistsvhntext: dataset, networks and running files for the MNIST-SVHN-TEXT dataset.
- celeba: dataset, networks and running files for the CelebA dataset.
- muse_eicu: dataset, networks and running files for the eICU dataset.
- modalities: modality interface of the datasets.
- eval_metrics: evaluation of the model, including reconstruction MSE, generation coherence, log-likelihood and latent space classification.
- run_script/run_epochs_vaes.py: training script of all methods for all the datasets.
- utils: needed function for the implementation.
the needed libraries are in requirements.txt.
The preparation of the PolyMNIST and MNIST-SVHN-TEXT datasets follows MoPoE.
Based on the MoPoE, the images of CelebA is processed to extract edges using the Canny algorithm, and the face segmentation map is generated by the Face Parsing Model trained on CelebM from FaRL.
eICU data set is not directly available and detailed description is in the paper 'Benchmarking machine learning models on eICU critical care dataset'.
After obtaining the eICU data through application, it can be processed using MUSE, and then the most frequently occurring parts from diagnosis, treatment, and medication are selected until the cumulative probability reaches 90%.
All launch scripts are located in the scripts folder, where run_{dataset_name}.sh corresponds to CLAP and run_{dataset_name}_graph.sh corresponds to GRAPHPL. To reproduce other methods, you need to modify run_{dataset_name}.sh to the corresponding impute_method (poe for MVAE, moe for MMVAE, joint_elbo for MoPoE-VAE) and set k_single to 0.
If you have any questions, please contact huxingjian@pku.edu.cn.
We would like to thank the authors of MoPoE, MUSE, FaRL, and CLAP for their valuable contributions.