FE-DeTr: Keypoint Detection and Tracking in Low-quality Image Frames with Events

Authors: Xiangyuanw Wang, Kuangyi Chen, Wen Yang, Lei Yu, Yannan Xing, Huai Yu

Introduction

FE-DeTr includes a novel keypoint detection network that fuses the textural and structural information from image frames with the high-temporal-resolution motion information from event streams. The network leverages a temporal response consistency for supervision, ensuring stable and efficient keypoint detection. Moreover, we use a spatio-temporal nearest-neighbor search strategy for robust keypoint tracking.

Network Architecture