Question about reproducing test accuracy on citation datasets

[Youth-49]

Hi, I have read your paper, downloaded the code, and ran it without any changes (I follow the same hyperparameters and random seeds). But I found that I can't reproduce the accuracy reported in the README.md file.

My results (dataset, test accuracy) are:

cora, 82.4%
citeseer, 73.0%
pubmed, 80.0%

My version:

PyTorch=1.7.1 with cuda
networkx=1.11
numpy=1.19.2
scipy=1.5.2
scikit-learn=0.24.1
hyperopt=0.1.1

did I miss some information or do something wrong?

[allenhaozhu]

Sorry to confuse you, you could check issues. I give different ways for this. Tips: SGC and SSGC are very sensitive to weights decays.......

[Youth-49]

Thanks for your reply.

yes I have checked the issues but I only found @EdisonLeeeee's experiments results, which are based on wrong codes (I guess) according to the following comments. (1-$\alpha$) shouldn't be multiplied with the term torch.spmm(adj, features). This bug has been fixed by you at that time and now I believe the codes are consistent with the paper.

I agree that they are sensitive to weight_decay. do you mean that the codes in this repository do not provide the right weight_decay parameter to achieve the test accuracy reported in the README file? I did not change the code provided in this repository.

Actually, I've tried several weight_decay values in SGC and SSGC, and I found that the best result SSGC can achieve on Cora is 82.8% under the weight_decay is {2e-5, 3e-5}, which is still below the accuracy reported.

if you change other parameters such as lr, please tell me because these parameters in the provided codes hold unchanged.

Thanks in advance.

[allenhaozhu]

In issue#13, I give a initialization for 83.5% in cora. Like what I explain, after fixed the bug, the performance slightly drop but the bug only on the cora,pubmed and citeseer because of deep copy in pytorch.