Map Prediction Data Collection

[Jaraxxus-Me]

Hi,
I see in the collect_mp.py file you are using the pre-trained MaskRCNN to collect semantic maps for map prediction.
May I know if you tried to use ground truth segmentation (habitat-sim semantic sensor) for the map data?
Thanks

[ajzhai]

If you use the ground truth for training input, there will be some distribution shift when you evaluate without it. A possibly better idea is that you can collect both the MaskRCNN map and the ground-truth map together, and train to take the (partial) MaskRCNN map as input but output the (complete) ground-truth map probability, so the prediction model can try to correct some MaskRCNN errors. I tried this briefly, but did not get any improvement. However, maybe it will work with a different architecture/loss.