How to scale cosine learning rate for small batchsize

[bowenc0221]

When training ResNet on ImageNet, the GluonCV uses a batchsize of 128x8GPUs with cosine learning rate. Do you have any suggestion to scale learning rate when using a small batchsize? I tried directly scaling down learning rate by the ratio of batchsize but it did not work very well.

Thanks.

[zhanghang1989]

GluonCV use fp16 training, 128 batch size per gpu. You can use smaller batch size, but should be at least 256 for total mini-batch size to reproduce compatible performance.