OAFuser： Towards Omni-Aperture Fusion for Light Field Semantic Segmentation
🛠️ 👷 🚀
🔥 We will release code and checkpoints in the future. 🔥
- 2023.10.08 Given that we are still conducting further research on Light Field Scene Understanding, the code will be released in subsequent versions. (Any questions, please contact the email at tengfei_tf@outlook.com.)
- 2023.09.25 Codestuff is on processing.
- 2023.07.29 Init repository.
- 2023.07.31 Release the arXiv version.
- Release the arXiv version.
- Code release.
Light field cameras can provide rich angular and spatial information to enhance image semantic segmentation for scene understanding in the field of autonomous driving. However, the extensive angular information of light field cameras contains a large amount of redundant data, which is overwhelming for the limited hardware resource of intelligent vehicles. Besides, inappropriate compression leads to information corruption and data loss. To excavate representative information, we propose an Omni-Aperture Fusion model (OAFuser), which leverages dense context from the central view and discovers the angular information from sub-aperture images to generate a semantically-consistent result. To avoid feature loss during network propagation and simultaneously streamline the redundant information from the light field camera, we present a simple yet very effective Sub-Aperture Fusion Module (SAFM) to embed sub-aperture images into angular features without any additional memory cost. Furthermore, to address the mismatched spatial information across viewpoints, we present Center Angular Rectification Module (CARM) realized feature resorting and prevent feature occlusion caused by asymmetric information.
(Overview)
Feel free to contact me if you have additional questions or have interests in semantic segmentation based on light field camera. Please drop me an email at tengfei_tf@outlook.com