Meta-Learning for Adaptation of Deep Optical Flow Networks

This is the official implementation of the MLOF (Meta Learning for Optical Flow) framework:

Meta-Learning for Adaptation of Deep Optical Flow Networks
Chaerin Min, Tae Hyun Kim, Jongwoo Lim
Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), 2023.
Paper / Video / Poster

TL;DR: We present an instance-wise meta-learning algorithm for optical flow domain adaptation

News

2023/11/29 - We released the MLOF 1.0.

Installation

Dataset

• FlyingChairs
• FlyingThings3D
• Sintel
• KITTI

├── datasets
    ├── Sintel
        ├── training
    ├── KITTI
        ├── training
    ├── FlyingChairs
        ├── data
    ├── FlyingThings
        ├── frames_cleanpass
        ├── frames_finalpass
        ├── optical_flow

Pre-trained model

Pre-trained checkpoint $\theta _0$ is given in ./checkpoints/gma-thing.pth

However, if you want to pre-train the optical flow model from scratch, you can do it:

sh bash/pretrain.sh

Meta-Train

sh bash/meta_train.sh

Meta-Inference

sh bash/meta_inference.sh

Citation

@inproceedings{min2023meta,
    title={Meta-Learning for Adaptation of Deep Optical Flow Networks},
    author={Min, Chaerin and Kim, Taehyun and Lim, Jongwoo},
    booktitle={Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision},
    pages={2145--2154},
    year={2023}
}

Related Resources

The overall code framework is adapted from GMA: Learning to Estimate Hidden Motions with Global Motion Aggregation and RAFT: Recurrent All Pairs Field Transforms for Optical Flow.