HolonNets are machine learning models that extend spectral convolutions to directed graphs. This repository contains the official implementation of the paper "HoloNets: Spectral Convolutions do extend to Directed Graphs", where we introduce -among other things - the FaberNet architecture which conforms to our general developed theory of HoloNets and achieves improved performance for node classification on heterophilic graphs.
Within the graph learning community, conventional wisdom dictates that spectral convolutional networks may only be deployed on undirected graphs: Only there could the existence of a well-defined graph Fourier transform be guaranteed, so that information may be translated between spatial- and spectral domains. Here we show this traditional reliance on the graph Fourier transform to be superfluous and -- making use of certain advanced tools from complex analysis and spectral theory -- extend spectral convolutions to directed graphs. We provide a frequency-response interpretation of newly developed filters, investigate the influence of the basis used to express filters and discuss the interplay with characteristic operators on which networks are based. In order to thoroughly test the developed theory, we conduct experiments in real world settings, showcasing that directed spectral convolutional networks provide new state of the art results for heterophilic node classification on many datasets and -- as opposed to baselines -- may be rendered stable to resolution-scale varying topological perturbations.
To get up and running with the project, you need to first set up your environment and install the necessary dependencies. This guide will walk you through the process step by step.
The project is designed to run on Python 3.10. We recommend using Conda to set up the environment as follows:
conda create -n holonet python=3.10
conda activate holonet
Once the environment is activated, install the required packages:
conda install pytorch==2.0.1 pytorch-cuda=11.7 -c pytorch -c nvidia
conda install pyg pytorch-sparse -c pyg
pip install ogb==1.3.6
pip install pytorch_lightning==2.0.2
pip install gdown==4.7.1
Please ensure that the version of pytorch-cuda
matches your CUDA version. If your system does not have a GPU, use the following command to install PyTorch:
conda install pytorch==2.0.1 -c pytorch
For M1/M2/M3 Mac users, pyg
(PyTorch Geometric) needs to be installed from source. Detailed instructions for this process can be found here.
-
run.py
: This script is used to run the models. -
model.py
: Contains the definition of the directed convolutional network called HoloNet in the code and FaberNet in the Paper.
This section provides instructions on how to reproduce the experiments outlined in the paper. Note that some of the results may not be reproduced exactly, given that some of the operations used are intrinsically non-deterministic on the GPU, as explained here. However, you should obtain results very close to those in the paper.
To reproduce the results of Table 1 in the paper, use the following command:
python -m src.run --dataset chameleon --use_best_hyperparams --num_runs 10
The --dataset
parameter specifies the dataset to be used. Replace chameleon
with the name of the dataset you want to use.
The following command line arguments can be used with the code:
Argument | Type | Default Value | Description |
---|---|---|---|
--dataset | str | "chameleon" | Name of the dataset |
--dataset_directory | str | "dataset" | Directory to save datasets |
--checkpoint_directory | str | "checkpoint" | Directory to save checkpoints |
Argument | Action | Description |
---|---|---|
--undirected | store_true | Use undirected version of graph |
--self_loops | store_true | Add self-loops to the graph |
--transpose | store_true | Use transpose of the graph |
Argument | Type | Default Value | Description |
---|---|---|---|
--model | str | "gnn" | Model type |
--hidden_dim | int | 64 | Hidden dimension of model |
--num_layers | int | 3 | Number of GNN layers |
--dropout | float | 0.0 | Feature dropout |
--alpha | float | 0.5 | Direction convex combination params |
--learn_alpha | action | - | If set, learn alpha |
--conv_type | str | "fabernet" | Model |
--normalize | action | - | If set, normalize |
--jk | str | "max" | Either "max", "cat" or None |
--weight_penalty | str | "exp" | Either "exp", "line" or None |
--k_plus | int | 3 | Polynomial Order |
--exponent | float | -0.25 | normalization in adj-matrix |
--lrelu_slope | float | -1 | LeakyRelu slope |
--zero_order | bool | False | Whether to use zero-order term |
Argument | Type | Default Value | Description |
---|---|---|---|
--lr | float | 0.001 | Learning Rate |
--weight_decay | float | 0.0 | Weight decay (if only real weights are used) |
--imag_weight_decay | float | 0.0 | Weight decay for imaginary part of weight matrices |
--real_weight_decay | float | 0.0 | Weight decay for real part of weight matrices |
--num_epochs | int | 10000 | Max number of epochs |
--patience | int | 10 | Patience for early stopping |
--num_runs | int | 1 | Max number of runs |
Argument | Type | Default Value | Description |
---|---|---|---|
--use_best_hyperparams | flag | If specified, use the best hyperparameters | |
--gpu_idx | int | 0 | Indexes of GPU to run the program on |
--num_workers | int | 0 | Number of workers for the dataloader |
--log | str | "INFO" | Log Level. Choices: ["DEBUG", "INFO", "WARNING"] |
--profiler | flag | If specified, enable profiler |
This repository builds on top of Emanuele Rossi's dir-gnn repository.
@inproceedings{
koke2024holonets,
title={HoloNets: Spectral Convolutions do extend to Directed Graphs},
author={Christian Koke and Daniel Cremers},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=EhmEwfavOW}
}
If you have any questions, issues or feedback, feel free to reach out to Christian Koke at christian.koke@tum.de
.