[CVPR 2024]. Efficient Meshflow and Optical Flow Estimation from Event Cameras. [Paper].

Xionglong Luo^1,4, Ao Luo^2,4, Zhengning Wang¹, Chunyu Lin³, Bing Zengn¹, Shuaicheng Liu^1,4

1.University of Electronic Science and Technology of China

2.Southwest Jiaotong University, 3.Beijing Jiaotong University, 4.Megvii Technology

Environments

You will have to choose cudatoolkit version to match your compute environment. The code is tested on Python 3.7 and PyTorch 1.10.1+cu113 but other versions might also work.

conda create -n EEMFlow python=3.7
conda activate EEMFlow
conda install pytorch==1.10.1 torchvision==0.11.2 torchaudio==0.10.1 cudatoolkit=11.3 -c pytorch -c conda-forge
pip install -r requirements

Dataset

MVSEC

You need download the HDF5 files version of MVSEC datasets. We provide the code to encode the events and flow label of MVSEC dataset.

# Encoding Events and flow label in dt1 setting
python loader/MVSEC_encoder.py --only_event -dt=1
# Encoding Events and flow label in dt4 setting
python loader/MVSEC_encoder.py --only_event -dt=4
# Encoding only Events
python loader/MVSEC_encoder.py --only_event

HREM

This work proposed a large-scale High-Resolution Event Meshflow (HREM) dataset (HREM), you can download it from https://pan.baidu.com/s/1iSgGCjDask-M_QqPRtaLhA?pwd=z52j .

Evaluate

Pretrained Weights

Pretrained weights can be downloaded from Google Drive. Please put them into the checkpoint folder.

Test on HREM

python test_EEMFlow_HREM.py -dt dt1
python test_EEMFlow_HREM.py -dt dt4

Citation

If this work is helpful to you, please cite:

@inproceedings{luo2024efficient,
  title={Efficient Meshflow and Optical Flow Estimation from Event Cameras},
  author={Luo, Xinglong and Luo, Ao and Wang, Zhengning and Lin, Chunyu and Zeng, Bing and Liu, Shuaicheng},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  pages={19198--19207},
  year={2024}
}

Acknowledgments

Thanks the assiciate editor and the reviewers for their comments, which is very helpful to improve our paper.

Thanks for the following helpful open source projects:

ERAFT, TMA, ADMFlow.