Skip to content

Latest commit

 

History

History
 
 

GNNAutoScale

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
examples
examples
 
 
gnn_models
gnn_models
 
 
README.md
README.md
 
 
dataset.py
dataset.py
 
 
graph_partition.py
graph_partition.py
 
 
history.py
history.py
 
 
utils.py
utils.py
 
 

README.md

GNNAutoScale: Auto-Scaling GNNs in PaddlePaddle

Refer to paper GNNAutoScale: Scalable and Expressive Graph Neural Networks via Historical Embeddings, we reproduce the GNNAutoScale framework using PGL, which can scale arbitrary message-passing GNNs to large graphs.

By using history embeddings on CPU to store updated node-embeddings from prior training iterations, and pulling neighboring node-embeddings in history embeddings to participate in training, we can have smaller GPU memory consumption.

Requirements

  • paddlepaddle-gpu>=2.2.0
  • pgl

Models

  • GCN
  • GAT
  • APPNP
  • GCNII

Commands

cd examples/

# GCN
python run.py --conf config/cora/gcn.yaml
python run.py --conf config/pubmed/gcn.yaml
python run.py --conf config/citeseer/gcn.yaml
python run.py --conf config/reddit/gcn.yaml
python run.py --conf config/arxiv/gcn.yaml

# GAT
python run.py --conf config/cora/gat.yaml
python run.py --conf config/pubmed/gat.yaml
python run.py --conf config/citeseer/gat.yaml

# APPNP
python run.py --conf config/cora/appnp.yaml
python run.py --conf config/pubmed/appnp.yaml
python run.py --conf config/citeseer/appnp.yaml

# GCNII
python run.py --conf config/cora/gcnii.yaml
python run.py --conf config/pubmed/gcnii.yaml
python run.py --conf config/citeseer/gcnii.yaml
python run.py --conf config/reddit/gcnii.yaml
python run.py --conf config/arxiv/gcnii.yaml

Results

Citation Network

Cora
Accuracy GNNAutoScale PyGAS PGL
Partition Method Metis Random Metis -
GCN 81.4(1.15) 78.12(1.28) 82.29(0.76) 80.7(1.0)
GAT 82.93(1.27) 82.79(0.99) 83.32(0.62) 83.4(0.4)
APPNP 83.94(0.55) 79.62(1.69) 83.19(0.58) 84.6(0.3)
GCNII 83.91(0.54) 81.36(1.26) 85.52(0.39) 84.6(0.3)
Citeseer
Accuracy GNNAutoScale PyGAS PGL
Partition Method Metis Random Metis -
GCN 70.86(0.93) 71.05(0.98) 71.18(0.97) 71.0(0.7)
GAT 71.56(0.9) 71.5(0.65) 71.86(1.00) 70.0(0.6)
APPNP 71.58(0.53) 70.35(1.3) 72.63(0.82) 71.9(0.3)
GCNII 70.06(1.11) 70.96(1.08) 73.89(0.48) 72.4(0.6)
Pubmed
Accuracy GNNAutoScale PyGAS PGL
Partition Method Metis Random Metis -
GCN 79.13(0.59) 78.47(0.44) 79.23(0.62) 79.4(0.3)
GAT 77.76(0.45) 77.4(0.56) 78.42(0.56) 77.2(0.4)
APPNP 80.4(0.28) 78.96(0.58) 79.82(0.52) 80.3(0.2)
GCNII 79.93(0.48) 78.51(0.49) 80.19(0.49) 79.8(0.3)

Reddit Dataset

Accuracy GNNAutoScale PyGAS
Partition Method Metis Random Metis
GCN 95.27 95.21 95.45
GCNII 96.72 96.71 96.77

Ogbn-arxiv Dataset

Accuracy GNNAutoScale PyGAS
Partition Method Metis Random Metis
GCN 71.8 70.7 71.68
GCNII 72.4 71.5 73.0

Coming new features

  • Multi-GPUs support.
  • Large dataset ogbn-papers100m support.

Reference

@inproceedings{Fey/etal/2021,
  title={{GNNAutoScale}: Scalable and Expressive Graph Neural Networks via Historical Embeddings},
  author={Fey, M. and Lenssen, J. E. and Weichert, F. and Leskovec, J.},
  booktitle={International Conference on Machine Learning (ICML)},
  year={2021},
}

Github link: https://github.com/rusty1s/pyg_autoscale