Question about batch_size, epoch setting

[guvcolie]

Hi, I'm trying to train DenseNet-121 on ImageNet dataset, but the result is poor...
Now I wander how to calculate the batch_size in muiti GPUs. You said "It took us 10 days to train 40M densenet for 120 epochs on 4 TITAN X GPUs, with batchsize 128" on issue (https://github.com/liuzhuang13/DenseNet/issues/5), you mean "each GPU use batchsize 128" or "each GPU use 32, sum is 128"?
Thank you!

[liuzhuang13]

Thanks and we meant "each GPU use 32, sum is 128".

[Jianf-Wang]

so... if I set 2 gpus, and batch_size =64, in fact, it is batchsize 128 ?

[liuzhuang13]

Actually the "batch_size" in the code means the total batch size. So if you want in total batch size 128 just set batch_size = 128.