Cannot reproduce best results

[NIRVANALAN]

Hi, I am trying to reproduce the best results of mag_cs dataset which achieves 50.2 in the paper. I adopt the same settings and early_stop as mentioned in your paper, but my best result achieved is 46.0 and generally the NMI score is around 43;
Besides, the weight_decay value in interactive.ipynb is 1e-5 by default while in your paper the regularization strength lambda=1e-2(which resulted poor performance). Which one should I use to reproduce the best result of CS dataset?

any suggestions?

[shchur]

Hi, thanks for your interest in our work.

The experiments in the paper have been done using the Tensorflow implementation, link to which you can find in README.md.

Regarding regularization: by default, the current version of the notebook uses a slightly different GCN implementation (called ImprovedGCN). This version doesn't use batch norm, and I empirically found it to work well with weight_decay = 1e-5.

The architecture described in the paper (and the one used in the original TF implementation) is based on vanilla GCN with batch normalization. For that model you should use weight_decay = 1e-2. To use the old architecture, you should uncomment the respective line in cell 5 of the notebook.
It should be possible to reproduce the results using the GCN architecture with weight_decay = 1e-2, since that is the architecture used in the paper.

To be completely honest, I haven't thoroughly tested the Pytorch implementation in this repository - I just created it in the process of learning Pytorch. I hope that I haven't introduced any serious bugs in the process of migrating from TF. Please let me know if you still reproduce the results using the vanilla GCN, and then I will have a look into the code.

[shchur]

After fixing the bug that you mentioned in #2 everything seems to work as expected.

[NIRVANALAN]

WOW, great!
Thanks for your work again