[question] About batch_size and learning rate in references/detection/train.py?

[wangg12]

Hi @fmassa ,
In references/detection/train.py, does the batch_size refer to images_per_gpu?
And is lr corresponding to the total batch size images_per_gpu x $NGPU?

Accordingly, if I want to use the linear scaling method for setting the learning rate for single GPU, should I set args.lr = lr / $NGPU and keep args.batch_size = images_per_gpu?

[fmassa]

In references/detection/train.py, does the batch_size refer to images_per_gpu?
And is lr corresponding to the total batch size images_per_gpu x $NGPU?

@wangg12 batch_size is indeed images_per_gpu.
The lr is the final lr that should be used, which corresponds to the lr for the images_per_gpu x n_gpu batches indeed.

Accordingly, if I want to use the linear scaling method for setting the learning rate for single GPU, should I set args.lr = lr / $NGPU and keep args.batch_size = images_per_gpu?

Yes, if you want to use a single GPU for training, you should modify the lr to be lr / 8, as the default learning rate value assumes 8 GPUs.

It would be great to have a better explanation of those hyperparameters, could you send a PR adding some comments to the code?

[wangg12]

Thanks, I can send a PR to explain them better in the code.