This is the repository for reproducing the results in our paper: [Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction] for the combination of GIANT-XRT+RevGAT.
Please follow the instruction in [GIANT] to get the GIANT-XRT node features. Note that if you generate your own pretrained node features from GIANT-XRT, you should be aware of your save path and modify the --pretrain_path (below in Step 3) accordingly.
After following the steps in [GIANT], go to the folder
pecos/examples/giant-xrt
Then git clone this repo in the folder giant-xrt directly.
If you install and run GIANT correctly, you should only need to additionally install dgl>=0.5.3. See here for pip/conda installation instruction for dgl.
Go to the folder deep_gcns_torch.
Run Runexp_RevGAT_ogbnarxiv.sh for reproducing our results for ogbn-arxiv dataset with GIANT-XRT features.
New arguments
--data_root_dir: path to save ogb datasets.
--pretrain_path: path to load GIANT-XRT features. Set it to 'None' for using ogb default features.
If execute correctly, you should have the following performance (using our pretrained GIANT-XRT features).
| RevGAT | RevGAT+KD | |
|---|---|---|
| Average val accuracy (%) | 77.01 ± 0.09 | 77.16 ± 0.09 |
| Average test accuracy (%) | 75.90 ± 0.19 | 76.15 ± 0.10 |
Number of params: 1304912
Remark: We do not carefully fine-tune RevGAT for our GIANT-XRT. It is possible to achieve higher performance by fine-tune it more carefully.
For more details about RevGAT, please check the original README.
If you find our code useful, please cite both our GIANT paper and the RevGAT references provided in the original repo.
Our GIANT paper:
@inproceedings{chien2021node,
title={Node Feature Extraction by Self-Supervised Multi-scale Neighborhood Prediction},
author={Chien, Eli and Chang, Wei-Cheng and Hsieh, Cho-Jui and Yu, Hsiang-Fu and Zhang, Jiong and Milenkovic, Olgica and Dhillon, Inderjit S},
booktitle={International Conference on Learning Representations (ICLR)},
year={2022}
}
RevGAT paper:
@InProceedings{li2021gnn1000,
title={Training Graph Neural Networks with 1000 layers},
author={Guohao Li and Matthias Müller and Bernard Ghanem and Vladlen Koltun},
booktitle={International Conference on Machine Learning (ICML)},
year={2021}
}
Feel free to email me (ichien3@illinois.edu) if you have further questions. My notification of the Github issue does not work properly so make sure to drop me an email when you open a new issue.