Question about 11th - Pooling block order

[A-suozhang]

Thank you for your great work, it really helped me a lot. However, I'm a bit confused about the 11th tip, which reorders the "conv-bn-relu-pooling" to "conv-polling-relu-bn". Is there a paper reference for this method, or other insight about why this would work?

[HolmesShuan]

Thanks for your interest. Conv-BN-ReLU-Pooling, Conv-Pooling-ReLU-BN and Conv-Pooling-BN-ReLU mostly come from our practical experience. It is hard to justify which one is better than the others. It depends on your tasks and backbone networks.

The reference can be found in "XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networks" section 4.3 "Ablations Studies".