Problem about BEV render

[zjulabwjt]

hi, @DRosemei thanks for your great work! I use my datasets run RoMe, but find in bev render, img is not a bev result. What wrong might cause this problem? Thanks for your help!

[DRosemei]

@zjulabwjt Sorry for late reply. You can check ego car poses and make sure they are mostly paralleled to road/ground.

[zjulabwjt]

@zjulabwjt Sorry for late reply. You can check ego car poses and make sure they are mostly paralleled to road/ground.

@DRosemei thanks for your reply. I visualize my ego pose xyz,The range of z-axis variation of the vehicle's pose is 4.9 meter during movement. And the trajectory nearly a straight line. I also found the trajectory before and after multiplying the vehicle's pose by the transform_normal2origin matrix does not match. Is it because the plane fitting algorithm fails in this case? Besides, how to understand the code transform_normal2origin[2, 3] += self.camera_height and in my dataset how to set this value, because if self.camera_height is not suitable, the loss nearly zero, and render img is white.

[DRosemei]

@zjulabwjt
Q: Is it because the plane fitting algorithm fails in this case?
A: It may be yes. Plane fitting algorithm is not always good especially. You can skip this and "camera_height", use actual (but may not be accurate road elevations) to init/supervise/fix elevation mlp.

[zjulabwjt]

@zjulabwjt Q: Is it because the plane fitting algorithm fails in this case? A: It may be yes. Plane fitting algorithm is not always good especially. You can skip this and "camera_height", use actual (but may not be accurate road elevations) to init/supervise/fix elevation mlp.

@DRosemei
Thank you for your advice. Following your suggestions, I improved the effectiveness of road surface reconstruction using pre-trained z mlp. However, In our scenario, there are a large number of vehicles. Due to the removal of dynamic objects during the training process, this results in numerous holes in the final road surface. I attempted to fuse the depth map from LiDAR with multi-view camera data, but I'm still encountering this issue. Is there any effective method to address this situation?

[DRosemei]

@DRosemei It is unavoidable, so we use more than one clip to reconstruct the whole area

[zjulabwjt]

@DRosemei It is unavoidable, so we use more than one clip to reconstruct the whole area

Thanks for your reply! Does multiple clip data refer to data collected multiple times from the same area at different times?

[DRosemei]

@zjulabwjt Yes, it does and you can refer https://www.bilibili.com/video/BV1ve4y1i7aZ?p=18&vd_source=5051310ed13090afc35ea319bbc5cac3 for more infomation.