About learning rate setting

[bityangke]

Hi Yi,
How did you decide the lr step ?
Did you follow somewhere else or experiment it youself ?
Thanks in advance!

[bryanyzhu]

Hi, I decided the lr step according to this paper.
But usually in my experiments, I just see when the loss/accuracy saturate, and then decay the lr. I find it more effective.

[bityangke]

Thanks very much.

[bityangke]

Hi Yi
I noted that Yuanjun used a batchsize（16x4，16 samples on each card） and iter size 4， so the “batch size” is equal to 256. They used step size 4000, 8000, 10000 （iters） , the corresponding batch steps should be 1000，2000，2500， and epoch number should be about 27，54， 67.
Am I right？
Thanks！

[bryanyzhu]

Hi, I am so sorry for the slow response, I am at CVPR this week.

I am also using similar strategy iter_size at here. So my corresponding batch steps are still 4000, 8000 and 10000, which is about 100, 200 and 250. Hope this is clear.

[bityangke]

Thank you！
Hope you have happy CVPR days！