SynchronizedBatchNorm2d or nn.BatchNorm2d?

[BinhuiXie]

What's the difference between SynchronizedBatchNorm2d and nn.BatchNorm2d? I usually freeze BN since the batch_size is usually small, i.e., 1. Dose this training trick you used could bring boost to your final performance. I think a comparison in fair settings will show the real improvement of your method.

[panzhang0212]

Previous UDA works use different batch size, 1, 2, 4, 8, 9. So, it's hard to set the same bs with all previous works. And we set batch size 4 due to 4 GPUs and do not try other batch sizes. If you want to try other bs, please choose a proper learning rate.