How important is remission/how is it used?

[ivannson]

I was wondering on the importance of the remission (or reflectivity) data? I have simulated Lidar data from Unreal Engine which I am planning to use to train a network from scratch, however that only provides X, X and Z coordinates.

Can I get rid of remission completely in the data preprocessing scrip/make remission, say, 0.5 for every point?

Alternatively I could assign a set value to each object ID, for example buildings will have a fixed remission r1, cars will have a fixed remission r2 etc.

Or will doing this be detrimental to the performance? Would be good to find out before spending a few days on data collection/preprocessing and then further 3-4 days on model training.

Thanks in advance!

[tano297]

Hi,

Remission seems to be very useful for the accuracy of the classification. However, if you will be rendering from unreal engine it will be really hard to get.

It all depends on how you will be inferring from the data. Will it be using real lidar scans? If this is the case, the most important thing is to figure out how to make the distribution of the input during inference be similar to the fake training data. SqueezeSeg V1 and V2 papers are a good starting point to check what to do when using simulated data.

No matter what approach you take with the remissions, you need to make sure that you can generate the data using the real sensor data you will use later, so the approach that has different remissions per class will not work. During inference, if you knew this beforehand then your classifier would be a simple lookup table and the cnn would not be needed, so you either need to simulate the remissions or set them to a fake constant value (like 0)

[ivannson]

Hi,

Thanks for the very clear explanation! The inference will be from simulated data as well (so trained on simulated data and tested on simulated data). I have read through both SqueezeSeg papers, which are indeed very useful. It seems that the lack of remission affects the accuracy when the model is trained on simulated data, and then tested on real data, I will do as you suggested and set the remission to a fake constant and see how well the model performs.

Another thing that was mentioned in SqueezeSegV2 is how they modelled remission using regression with L2 loss. Since remission depends on the distance, angle and material, I'll try modelling remission based on KITTI data with x, y, z, distance and class as input features. Would this method have a similar risk of essentially becoming a look up table, and if the model performs significantly better with modelled remission, would there be a way to verify that it didn't overfit?

[tano297]

Sorry for the delay.
With simulated data it is always hard to say what the model is really doing, since the optimization is mostly going to try to find the easiest way to minimize the loss. So in order to check that your model did not overfit to a particular peculiarity of your simulated data (such as the remission in this case), I would suggest that you apply different levels of noise to that channel during both training and inference on the training and validation sets, and plot both results. Under significant amounts of noise in this channel, a non overfitted model should perform reasonably well, while a "lookup-table-like" one will fail completely.
I will close this issue because it is not an issue with the code, but we can keep the chat going here!