How to repeat your best result in CityPerson？

[yitang-hnu]

I have tried to repeat your Cascade Mask-R-CNN with the pre-trained model and the config you provided for many times, but got a much higher MR than your benchmarking(about 14.74% in reasonable datasets). Could you share the details about how to repeat your best result in CityPerson?

[hasanirtiza]

What do you mean by you cannot repeat ? did you mean that you download the pre-trained model provided in the repo and simply ran evaluation script on CityPersons and you cannot get the performance as stated in the benchmarking table of the README.md ? or you are trying to train a model by yourself using a pre-trained model ?

Regrading the first part, downloading this model and simply running
python ./tools/test_city_person.py configs/elephant/cityperson/cascade_hrnet.py ./models_pretrained/epoch_ 5 6 --out result_citypersons.json --mean_teacher should give you LAMR Reasonable : 7.5 and Heavy : 27.4. This has been tested several times and on multiple machines.

If you are trying to train it yourself (second part), please read the ArXiv pre-print version, section 6.1 and 6.2 to understand how to train.

[yitang-hnu]

Thanks for your reply!
I can get the best result by running the evaluation script on CityPersons, but I want to train the model by myself to get your best result in CityPerson. From section 6.2 in your paper, did you first pretrain the Cascade R-CNN in the Wider Pedestrian, and then pretrain again in ECP, and finally finetune in CityPerson? If so, I want to know the details about the learning rate and the epoch during pretraining and finetuning.

[hasanirtiza]

If so, I want to know the details about the learning rate and the epoch during pretraining and finetuning.

Yes, as stated in the paper, we trained it using progressive training pipeline. You can start by using the same learning rate as provided in the configs (for example in the case of Cascade R-CNN 0.02 ->8 gpus with 2 imgs/gpu). For the first part of the pipeline(pre-training on say WiderPedestrian) 20 epochs are usually enough, during the middle part of the pipeline(ECP) you can usually increase the number of epochs (have a look at the validation error and see where it converges). Finally, in the last part, usually 5-7 epochs are enough for the model to converge. You can also experiment with the learning rate if you want, especially in the last part.

[yitang-hnu]

@hasanirtiza. Thanks for your detailed explanation for pre-training and fin-tuning! It really works, and I got the result of 8.53 MR^-2 in CityPerson, but it is still 1 MR-^2 point behind your best result. I have tried a series of learning rate(from 0.0025 to 0.0005 for 1 img and 1 GPU) for fine-tuning, and only got the result of 8.53 MR^-2 in CityPerson. So, I think it is not very well of my pre-trained model on Wilder Pedestrian and ECP before fine-tune. Could you provide the model that you pre-trained on Wilder Pedestrian and ECP before fine-tuning step in your paper Sec 6.2?

BTW, from my experiments on CityPerson datasets, the model pre-trained on Wilder Pedestrian only is much better than the model pre-trained on WIDER PERSON and WIDER Pedestrian that you provided on CityPerson datasets. If it is convenient, could you provide the pre-trained model only on WIDER Pedestrian? I think it is also much helpful for generic human detection.