Is Multi-RPN doing the right thing?

[kumatheworld]

This is more like a question to the paper of SiamRPN++.

To see how different layers contribute to the RPN part, I took a look at the pre-trained model of siamrpn_r50_l234_dwxcorr and got the following parameter values.

	layer 2	layer 3	layer 4
rpn_head.cls_weight	0.2020	0.3365	-0.5385
rpn_head.loc_weight	-0.4176	-0.4808	0.8984

This table gives me the impression that the classification task relies more on earlier layers and the localization task relies more on later layers. This is a little bit counterintuitive to me because later layers tend to lose spatial information to get more abstract features suitable for classification. Could anybody give me some explanations?

And I'm not sure if the aggregation method of Multi-RPN is legitimate. Each of the 2 branches is taking a weighted sum of RPN-ed features from different layers, but I don't think taking a weighted sum in the localization branch is very accurate because dw's and dh's are fed to exponential functions. Weighted sums of modified anchors would make sense but I suppose non-linearity of exponential functions might make localization inaccurate.