Fail to reproduce MOTA on MOT17

[withinnoitatpmet]

Hi,

Thank you for your wonderful work! I am trying to reproduce you tracktor++ result in MOT17 dataset. Using your trained weights, 61.6 MOTA was reproduced on mot17_train_FRCNN17 as described from paper. However, when I tried to train FPN and siamese network, I only get 49.8 MOTA on mot17_train_FRCNN17.

I noticed that the config.yaml file downloaded and the config file produced by training RPN are different. Here is part of the difference. It seems that parameters are saved in model.

---

< FEAT_STRIDE: &id012

FEAT_STRIDE: &id010
20c19
< FPN_ANCHOR_SCALES: &id013

---

FPN_ANCHOR_SCALES: &id011
27c26
< FPN_FEAT_STRIDES: &id014

---

FPN_FEAT_STRIDES: &id012
37c36
< MOBILENET: &id015 !!python/object/new:easydict.EasyDict

---

MOBILENET: &id013 !!python/object/new:easydict.EasyDict
48c47
< PIXEL_MEANS: &id016 !!python/object/apply:numpy.core.multiarray._reconstruct

---

PIXEL_MEANS: &id014 !!python/object/apply:numpy.core.multiarray._reconstruct

---

I evaluated the downloaded model and the model trained by myself, mAPs are roughly the same on val set(0.79x). However, the PIXEL MEANS and STDVS are different.

trained by myself:
'PIXEL_MEANS': array([[[123.675, 116.28 , 103.53 ]]]),
'PIXEL_STDVS': array([[[58.395, 57.12 , 57.375]]]),

downloaded from you(also same as pascal voc pre file):
'PIXEL_MEANS': array([[[102.9801, 115.9465, 122.7717]]]),
'PIXEL_STDVS': array([[[1., 1., 1.]]]),

Do you have any ideas?

[withinnoitatpmet]

Hi,

I find that the pixel_means and stds comes from src/fpn/fpn/model/fpn/resnet.py
347L
cfg.PIXEL_MEANS = 255 * np.array([[[0.485, 0.456, 0.406]]])
cfg.PIXEL_STDVS = 255 * np.array([[[0.229, 0.224, 0.225]]])
When using FPNResNet model, pixel means and stds will be hard coded other than load from cfg file. How can I reproduced your fpn model?

[withinnoitatpmet]

I trained a FPN model which is compatible to your release model.

The problem is that when pretrained is set True, RPN will try to download weights from aws and hard code pixel means and stvds.

Fix is simple as following:

+++ b/trainval_net.py
@@ -334,11 +334,11 @@ if name == 'main':

 # initilize the network here.
 if args.net == 'res101':

•    FPN = FPNResNet(imdb.classes, 101, pretrained=True)
  

•    FPN = FPNResNet(imdb.classes, 101, pretrained=False)
  

  elif args.net == 'res50':

•    FPN = FPNResNet(imdb.classes, 50, pretrained=True)
  

•    FPN = FPNResNet(imdb.classes, 50, pretrained=False)
  

  elif args.net == 'res152':

•    FPN = FPNResNet(imdb.classes, 152, pretrained=True)
  

•    FPN = FPNResNet(imdb.classes, 152, pretrained=False)
  

  else:
  print("Network is not defined.")
  pdb.set_trace()

[dichen-cd]

Hello @withinnoitatpmet . Have you reproduced the reported result successfully?