Questions about experimental results

[wuzeww]

Hi，

The results I got with the model you gave are different from those in your paper, and the results are different under different devices. I want to know if you have the same problem at that time. Is this because the test objects are randomly selected and randomly placed? Is the test scene generated each time not fixed, so sometimes the experimental results are different？

Looking forward to your answer. Thank you！

[Steve-Tod]

Hi, there is randomness in the torch_scatter package we used which can't be removed by setting random seeds. So even we use exactly the same scene setup, the results are still a little bit different. However, there should not be a large difference. I just ran two experiments and got the following results:

I use the same random seed here, but get different results which are both significantly higher than the number in the paper. This is because the number given in the paper is the average result of multiple runs with different random seeds (by specifying --seeds 1 2 3 4 5 in arguments). Different random seeds may give different levels of difficulty.

Also, what do you mean by different devices?

[wuzeww]

Thank you very much for your answer, I understand now.

My previous "device" means the result of running under different computers.

Is your seeds set like this? --num-rounds 100 --seeds [0, 1, 2, 3, 4]

[Steve-Tod]

I don't remember the exact random seeds I used, but you can try these first.

[wuzeww]

Ok, thank you!

[wuzeww]

Hi！

I have a new question. Are the epoch of GIGA, GIGA-Aff and VGN the same? Are they all 20？