original images as input

[jiayily]

Hello, it's very strange that you use image mask as the model's input. Have you ever tried input original images?

[bastilam]

In the paper, you find our reasoning for using segmented images as input:

Instead of trying to make simulated images look more realistic, we remove mostly unnecessary texture from real-world data by computing semantically segmented camera images. We show how their use as input to our algorithm allows us to train a neural network on synthetic data only, while still being able to successfully perform the desired task on real-world data.

The usage of synthetic datasets and an input abstraction to semantically segmented representations of the camera images allows the application to real-world data without manual labeling of BEV images.

Feel free to test the approach with original images.

[mikqs]

Hello,

Could you please explain how one can use real-world images after training the model ? I have tested the model successfully on the validation dataset from VTD, but for real-world data, I believe that I have to semantically segment it with the color palette that was used for training. Is there an existing model that you recommend for segmentation ? (i.e. which model do you use to label the left-most real-world input pictures of Fig. 6 from the paper ?)

Thank you