This is the official implementation of our ICCV2021 paper GyroFlow.
We also provide a MegEngine version, check at GyroFlow
Our presentation video: [Youtube][Bilibili].
- Requirements please refer to
requirements.txt.
2021.11.15: We release the GOF_Train V1 that contains 2000 samples.
The download link is GoogleDrive or CDN. Put the data into ./dataset/GOF_Train, and the contents of directories are as follows:
./dataset/GOF_Train
├── sample_0
│ ├── img1.png
│ ├── img2.png
│ ├── gyro_homo.npy
├── sample_1
│ ├── img1.png
│ ├── img2.png
│ ├── gyro_homo.npy
.....................
├── sample_1999
│ ├── img1.png
│ ├── img2.png
│ ├── gyro_homo.npy
For quantitative evaluation, including input frames and the corresponding gyro readings, a ground-truth optical flow is required for each pair.
The download link is GoogleDrive or CDN. Move the file to ./dataset/GOF_Clean.npy.
The most difficult cases are collected in GOF-Final.
The download link is GoogleDrive. Move the file to ./dataset/GOF_Final.npy.
To train the model, you can just run:
python train.py --model_dir experiments
Load the pretrained checkpoint and run:
python test.py --model_dir experiments/demo_experiment/exp_2 --restore_file experiments/demo_experiment/exp_2/test_model_best.pth
If you think this work is useful for your research, please kindly cite:
@InProceedings{Li_2021_ICCV,
author = {Li, Haipeng and Luo, Kunming and Liu, Shuaicheng},
title = {GyroFlow: Gyroscope-Guided Unsupervised Optical Flow Learning},
booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
month = {October},
year = {2021},
pages = {12869-12878}
}
In this project we use (parts of) the official implementations of the following works:
We thank the respective authors for open sourcing their methods.