Performance is unstable.

[AlmoonYsl]

When I use batch_size=2, num_gpus=8 for training stream3dppe, the performance is very low (~55 NDS), and when I use SyncBN the performance is still low (~57 NDS).
When I use batch_size=4, num_gpus=4, I can reproduce the result (58.37 NDS).
I am confused. Why the performance depends heavily on samples_per_gpu?

[drilistbox]

This is a normal phenomenon for petr-series method.
And the setting, i.e. (bs4 * 4gpus with BN) / (bs2 * 8gpus with BN) / (bs4 * 4gpus with SyncBN), are not equivalent. Parameters need to be adjusted under different settings.

[AlmoonYsl]

This is a normal phenomenon for petr-series method. And the setting, i.e. (bs4 * 4gpus with BN) / (bs2 * 8gpus with BN) / (bs4 * 4gpus with SyncBN), are not equivalent. Parameters need to be adjusted under different settings.

Thanks for your replay. If I want to use batch_size=2, num_gpus=8 for training stream3dppe, what the config should be changed? Can you give me some suggestions?

[drilistbox]

In fact, we have never conduct ablation study on stream3dppe with 8 gpus as the sources are limited.
I thought bs per gpu and lr are also need to be adapted. Best wishes!

[AlmoonYsl]

In fact, we have never conduct ablation study on stream3dppe with 8 gpus as the sources are limited. I thought bs per gpu and lr are also need to be adapted. Best wishes!

Thanks for your suggestions!