Evaluating graph neural nets with unseen nodes
Paper: L Galke, I Vagliano, A Scherp: Can Graph Neural Networks Go "Online"? An Analysis of Pretraining and Inference, Representation Learning on Graphs and Manifolds workshop, ICLR 2019.
A link to the paper will follow soon.
Accuracy scores after inserting previously unseen, unlabelled nodes and edges.
For the evaluate script: python3, torch, dgl, pandas
For the visualization script: matplotlib, seaborn
pip install -r requirements.txt
Reproducing our experiments
We provide a convenience script for re-generating all the results from the paper.
Running own experiments
The pretraining/inference experiments can be conducted with the
usage: evaluate.py [-h] [--dataset DATASET] [--model MODEL] [--runs RUNS] [--inference INFERENCE] [--invert] [--outfile OUTFILE] [--epochs EPOCHS] [--no-cuda] [--info] optional arguments: -h, --help show this help message and exit --dataset DATASET The input dataset. --model MODEL Specify model --runs RUNS Number of random reruns --inference INFERENCE Number of inference epochs --invert Invert train and test set --outfile OUTFILE Dump Results to outfile --epochs EPOCHS Number of training epochs --no-cuda Force no cuda
For batching experiments, see the example in
We provide a script to visualize the results generated by
The script should be called on the results file specified by
python3 visualize.py results.txt
Adding new models
New models should be implemented as subclasses of
nn.Module and implement
features are the node features. They
additionally need to satisfy
g is a
instance. This method will be called between the pretraining and the inference
epochs. They need to be registered along with hyperparameters and optimizers