awesome-efficient-gnns.md

🚀 Awesome Efficient Graph Neural Networks

This is a curated list of must-read papers on efficient Graph Neural Networks and scalable Graph Representation Learning for real-world applications. Contributions for new papers and topics are welcome!

Accompanying Blogpost: chaitjo.com/post/efficient-gnns

Efficient and Scalable GNN Architectures

• [ICML 2019] Simplifying Graph Convolutional Networks. Felix Wu, Tianyi Zhang, Amauri Holanda de Souza Jr., Christopher Fifty, Tao Yu, Kilian Q. Weinberger.
• [ICML 2020 Workshop] SIGN: Scalable Inception Graph Neural Networks. Fabrizio Frasca, Emanuele Rossi, Davide Eynard, Ben Chamberlain, Michael Bronstein, Federico Monti.
• [ICLR 2021 Workshop] Adaptive Filters and Aggregator Fusion for Efficient Graph Convolutions. Shyam A. Tailor, Felix L. Opolka, Pietro Liò, Nicholas D. Lane.
• [ICLR 2021] On Graph Neural Networks versus Graph-Augmented MLPs. Lei Chen, Zhengdao Chen, Joan Bruna.
• [ICML 2021] Training Graph Neural Networks with 1000 Layers. Guohao Li, Matthias Müller, Bernard Ghanem, Vladlen Koltun.

Source: Simplifying Graph Convolutional Networks

Neural Architecture Search for GNNs

• [IJCAI 2020] GraphNAS: Graph Neural Architecture Search with Reinforcement Learning. Yang Gao, Hong Yang, Peng Zhang, Chuan Zhou, Yue Hu.
• [AAAI 2021 Workshop] Probabilistic Dual Network Architecture Search on Graphs. Yiren Zhao, Duo Wang, Xitong Gao, Robert Mullins, Pietro Lio, Mateja Jamnik.
• [IJCAI 2021] Automated Machine Learning on Graphs: A Survey. Ziwei Zhang, Xin Wang, Wenwu Zhu.

Source: Probabilistic Dual Network Architecture Search on Graphs

Large-scale Graphs and Sampling Techniques

• [KDD 2019] Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks. Wei-Lin Chiang, Xuanqing Liu, Si Si, Yang Li, Samy Bengio, Cho-Jui Hsieh.
• [ICLR 2020] GraphSAINT: Graph Sampling Based Inductive Learning Method. Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, Viktor Prasanna.
• [CVPR 2020] L²-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks. Yuning You, Tianlong Chen, Zhangyang Wang, Yang Shen.
• [KDD 2020] Scaling Graph Neural Networks with Approximate PageRank. Aleksandar Bojchevski, Johannes Klicpera, Bryan Perozzi, Amol Kapoor, Martin Blais, Benedek Rózemberczki, Michal Lukasik, Stephan Günnemann.
• [ICML 2021] GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings. Matthias Fey, Jan E. Lenssen, Frank Weichert, Jure Leskovec.
• [ICLR 2021] Graph Traversal with Tensor Functionals: A Meta-Algorithm for Scalable Learning. Elan Markowitz, Keshav Balasubramanian, Mehrnoosh Mirtaheri, Sami Abu-El-Haija, Bryan Perozzi, Greg Ver Steeg, Aram Galstyan.

Source: GraphSAINT: Graph Sampling Based Inductive Learning Method

Low Precision and Quantized GNNs

• [EuroMLSys 2021] Learned Low Precision Graph Neural Networks. Yiren Zhao, Duo Wang, Daniel Bates, Robert Mullins, Mateja Jamnik, Pietro Lio.
• [ICLR 2021] Degree-Quant: Quantization-Aware Training for Graph Neural Networks. Shyam A. Tailor, Javier Fernandez-Marques, Nicholas D. Lane.
• [CVPR 2021] Binary Graph Neural Networks. Mehdi Bahri, Gaétan Bahl, Stefanos Zafeiriou.

Source: Degree-Quant: Quantization-Aware Training for Graph Neural Networks

Knowledge Distillation for GNNs

• [CVPR 2020] Distilling Knowledge from Graph Convolutional Networks. Yiding Yang, Jiayan Qiu, Mingli Song, Dacheng Tao, Xinchao Wang.
• [WWW 2021] Extract the Knowledge of Graph Neural Networks and Go Beyond it: An Effective Knowledge Distillation Framework. Cheng Yang, Jiawei Liu, Chuan Shi.
• [IJCAI 2021] On Self-Distilling Graph Neural Network. Yuzhao Chen, Yatao Bian, Xi Xiao, Yu Rong, Tingyang Xu, Junzhou Huang.
• [IJCAI 2021] Graph-Free Knowledge Distillation for Graph Neural Networks. Xiang Deng, Zhongfei Zhang.
• [ArXiv 2021] On Representation Knowledge Distillation for Graph Neural Networks. Chaitanya K. Joshi, Fayao Liu, Xu Xun, Jie Lin, Chuan-Sheng Foo.

Source: On Representation Knowledge Distillation for Graph Neural Networks

Hardware Acceleration of GNNs

• [IPDPS 2019] Accurate, Efficient and Scalable Graph Embedding. Hanqing Zeng, Hongkuan Zhou, Ajitesh Srivastava, Rajgopal Kannan, Viktor Prasanna.
• [IEEE TC 2020] EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks. Shengwen Liang, Ying Wang, Cheng Liu, Lei He, Huawei Li, Xiaowei Li.
• [FPGA 2020] GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms. Hanqing Zeng, Viktor Prasanna.
• [IEEE CAD 2021] Rubik: A Hierarchical Architecture for Efficient Graph Learning. Xiaobing Chen, Yuke Wang, Xinfeng Xie, Xing Hu, Abanti Basak, Ling Liang, Mingyu Yan, Lei Deng, Yufei Ding, Zidong Du, Yunji Chen, Yuan Xie.
• [ACM Computing 2021] Computing Graph Neural Networks: A Survey from Algorithms to Accelerators. Sergi Abadal, Akshay Jain, Robert Guirado, Jorge López-Alonso, Eduard Alarcón.

Source: Computing Graph Neural Networks: A Survey from Algorithms to Accelerators

Code Frameworks, Libraries, and Datasets

• [PyG] PyTorch Geometric.
• [DGL] Deep Graph Library.
• [NeurIPS 2020] Open Graph Benchmark: Datasets for Machine Learning on Graphs. Weihua Hu, Matthias Fey, Marinka Zitnik, Yuxiao Dong, Hongyu Ren, Bowen Liu, Michele Catasta, Jure Leskovec.
• [KDD Cup 2021] OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs Weihua Hu, Matthias Fey, Hongyu Ren, Maho Nakata, Yuxiao Dong, Jure Leskovec.
• [CIKM 2021] PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models. Benedek Rozemberczki, Paul Scherer, Yixuan He, George Panagopoulos, Alexander Riedel, Maria Astefanoaei, Oliver Kiss, Ferenc Beres, Guzmán López, Nicolas Collignon, Rik Sarkar.

Source: OGB-LSC: A Large-Scale Challenge for Machine Learning on Graphs

Industrial Applications and Systems

• [KDD 2018] Graph Convolutional Neural Networks for Web-Scale Recommender Systems. Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L. Hamilton, Jure Leskovec.
• [VLDB 2019] AliGraph: A Comprehensive Graph Neural Network Platform. Rong Zhu, Kun Zhao, Hongxia Yang, Wei Lin, Chang Zhou, Baole Ai, Yong Li, Jingren Zhou.
• [KDD 2020] PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest Aditya Pal, Chantat Eksombatchai, Yitong Zhou, Bo Zhao, Charles Rosenberg, Jure Leskovec.
• [CIKM 2020] P-Companion: A Principled Framework for Diversified Complementary Product Recommendation Junheng Hao, Tong Zhao, Jin Li, Xin Luna Dong, Christos Faloutsos, Yizhou Sun, and Wei Wang.
• [CIKM 2021] ETA Prediction with Graph Neural Networks in Google Maps. Austin Derrow-Pinion, Jennifer She, David Wong, Oliver Lange, Todd Hester, Luis Perez, Marc Nunkesser, Seongjae Lee, Xueying Guo, Brett Wiltshire, Peter W. Battaglia, Vishal Gupta, Ang Li, Zhongwen Xu, Alvaro Sanchez-Gonzalez, Yujia Li, Petar Veličković.

Source: Graph Convolutional Neural Networks for Web-Scale Recommender Systems