Label Images vs. Input Images

[rakshitagouni]

What is the difference between label and input images in the response I got from a prior issue?
If I'm using my own dataset with only front camera images as input, is the following organization correct?

• front folder: segmented images
• bev: ground truth BEV images on non-segmented data
• bev+occlusion: occlusion.py run on bev folder imgs
• homographies: ipm.py run on front folder imgs

to confirm, the segmented images do not go in the preprocessing scripts?

---

Problem is that your input image has a fourth alpha channel, s.t. the resized image has shape (256, 512, 4). This causes the crash during one-hot-encoding.

I will push a fix tomorrow, s.t. an image will always be loaded as RGB instead of RGBA, even if present. In the meantime, you can fix it yourself by replacing utils.py#L77 with

img = tf.image.decode_png(img, channels=3)

Some more notes on your files:

• The standard implementation expects semantically segmented input and output images, which are then one-hot-en/decoded as part of the pipeline. Your images are a blend of the real-world-image and the semantic segmentation. One-hot-en/decoding will not work properly this way.
• Your input image color-codes vehicles in a purple-ish way, but the standard 0,0,142 (RGB) blue is listed in the convert_10.xml. You need to check the colors you specify there.
• Your label image has shape (640, 480), while your input image has shape (480, 640). Keep in mind that both will be center-cropped/resized to (256, 512).
• It's important that you provide a good estimate of the homography matrix. Just saying as I couldn't have a look at your homography file.

Originally posted by @lreiher in #3 (comment)

[lreiher]

• front folder: segmented images

... from vehicle camera, yes. These are the input images.

• bev: ground truth BEV images on non-segmented data

On non-segmented data? This should simply be the GT BEV image, also semantically segmented.

• bev+occlusion: occlusion.py run on bev folder imgs

Yes. These are the label images.

• homographies: ipm.py run on front folder imgs

Yes, but not needed for uNetXST.

---

Please spend some more time studying the sample data that we provide and read our paper to get a decent understanding of what problem we are trying to solve. Your are, of course, free to extend our code to your needs, which might incorporate non-segmented images.