Explanation Accuracy Much Lower Than Table in Paper for syn2/syn4/syn5

[HuichenLi]

Dear authors,
Thank you for sharing the code. I am trying to run your code to reproduce the results reported in your paper. I downloaded the project, unzip it, and then run
python3 script_eval_gt.py --dataset synX
with X equals 1, 2, 4, and 5, without changing anything in the project. With the dataset syn1, I got node accuracy 0.99, which matches the number in Table 1. However, I got node accuracy 0.62/0.61/0.51 for syn2/syn4/syn5, which are much lower than the 0.93/0.97/0.93 in Table 1 for these three cases.
Did I misinterpret anything? Are there something special about syn2/syn4/syn5 datasets that I should modify before I can get the same results as Table 1? Thank you! Your response/help is greatly appreciated.

[AlexDuvalinho]

Dear writer, The configurations below may not be the exact final settings I regret (the ones yielding SOTA) but they should help you get good performance on these slightly more challenging datasets: If you are interested by 2-hops neighbours: python3 script_eval_gt.py --num_samples=65 --dataset='syn1' --hops=2 --cuda=1 --hv='compute_pred' --coal='SmarterSeparate' --feat='Expectation' --S=1 --fullempty=None python3 script_eval_gt.py --num_samples=100 --dataset='syn2' --hops=2 --hv='compute_pred' --coal='SmarterSeparate' --feat='All' --epochs=100 --g='WLR_sklearn' --S=1 --fullempty=None python3 script_eval_gt.py --num_samples=300 --dataset='syn4' --hops=2 --hv='compute_pred' --coal='SmarterSeparate' --feat='Expectation' --epochs=100 --g='WLR_sklearn' --S=4 --fullempty=None python3 script_eval_gt.py --num_samples=400 --dataset='syn5' --hops=2 --fullempty=None --hv='compute_pred' --S=4 --coal='SmarterSeparate' --feat=‘Expectation’ Considering 3 hops neighbours, keep the same settings as above and just replace the number of samples by: syn1: --num_samples=400 syn2: --num_samples=800 syn4: --num_samples=1400 syn5: --num_samples=1000 I hope this will solve your problem. Kind regards, The authors

…

On 25 Jan 2022, at 22:52, Huichen Li ***@***.***> wrote: Dear authors, Thank you for sharing the code. I am trying to run your code to reproduce the results reported in your paper. I downloaded the project, unzip it, and then run python3 script_eval_gt.py --dataset synX with X equals 1, 2, 4, and 5, without changing anything in the project. With the dataset syn1, I got node accuracy 0.99, which matches the number in Table 1. However, I got node accuracy 0.62/0.61/0.51 for syn2/syn4/syn5, which are much lower than the 0.93/0.97/0.93 in Table 1 for these three cases. Did I misinterpret anything? Are there something special about syn2/syn4/syn5 datasets that I should modify before I can get the same results as Table 1? Thank you! Your response/help is greatly appreciated. — Reply to this email directly, view it on GitHub <#1>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AKMBBLZEE2PTL3UQUAFEXD3UX4LRRANCNFSM5MZN3JOQ>. Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>. You are receiving this because you are subscribed to this thread.