Why you need to make divisor 16 when deploying model to mmdeploy

[twmht]

I am now reviewing the code of fisher pruning, and find something interesting (https://github.com/open-mmlab/mmrazor/blob/main/configs/pruning/mmdet/group_fisher/retinanet/group_fisher_act_deploy_retinanet_r50_fpn_1x_coco.py#L60).

Here you gave a divisor 16 when deploying, why you need to do that?

[LKJacky]

This is because of the characteristics of hardware, like GPU. These hardwares run faster if the dimension of a tensor/matrix is divisible by a constant, like 8, 16, or 32.

[twmht]

@LKJacky

yup. i knew that, and this is what i want. did you just append all zero weight convolutions at the end of the channels? why not just do that before finetuning? becuase i don't think it would be worse and can be optimized by tensorcore when training as well.

[LKJacky]

We have two reasons to do that when deploying rather than before finetuning:

1. Padding when deploying is easier than padding before finetuning. Because finetuning needs to consider the initialization of the padding values. When deploying, we only need to pad zero.
2. We guess different hardware may need different divisors. Padding when deploying is more flexible.

You absolutely can pad before finetune, and we also provide the choice for you, please refer to source code

[twmht]

appreciate for your reply. i need some time to read the code.