question about config under large net like Resnet152

[math-yyj]

For case like using Resnet-152, one single GPU can not have big batch
I am interseted in

(1)How many P40 GPUs are used when training with Res152,
(2)what is the total batch, and the batch on each GPU
(3)Did you use SyncBN across GPUs?

Many Thanks !

[lijiannuist]

Hi @mathyyj
(1) 2 P40 GPUs when training Res152.
(2) We set Batch size 16, and each GPU has 8 batch.
(3) We did not use SyncBN because we think 8 batch per device is usually large enough to obtain good BN statistics.
However, when we use 32 Batchsize and 4 P40 GPUs, we get better performance. SyncBn maybe work.

[math-yyj]

I see that, Thank you very much!