This project provides the PyTorch implementation of Group Whitening described in the following paper:
Group Whitening: Balancing Learning Efficiency and Representational Capacity, arXiv:2009.13333.
- We believe the proposed Group Whitening (GW) module is a practical component, the implementation of which is in the directory:
- GW consistently improves the performance of ResNet and ResNeXt, with absolute gains of 1.02% ∼ 1.49% in top-1 accuracy on ImageNet (repo:
./classification/) and 1.82% ∼ 3.21% in bounding box AP on COCO object detection (repo:
Table 1, ImageNet Classification. Standard setup: batchSize=256, wd=0.0001, init lr=0.1, 100 epochs (30,60,90- decay).
|BaseLine||Using GW module|
|ResNet-50||76.23||77.72 (model, pth )|
|ResNet-101||77.69||78.71 (model, pth )|
|ResNeXt-50||77.01||78.43 (model, pth )|
|ResNeXt-101||79.29||80.43 (model, pth )|
Table 2, COCO object detection results using Faster R-CNN with ResNet-50+FPN. We use the 1x lr scheduling (90k iterations), with a batch size of 16 on 8 GPUs.
|2fc head box||4conv 1fc head box|
|BN-frozen||36.31% AP||36.39 % AP|
|GN||36.32% AP||37.86 % AP|
|GW||38.13% AP||39.60 % AP|
Table 3, COCO object detection and segmentation results using Mask R-CNN with ResNeXt-101+FPN. We use the 1x lr scheduling (180k iterations), with a batch size of 8 on 8 GPUs.