Flow-Based Spatio-Temporal Structured Prediction of Motion Dynamics

Dependencies

• PyTorch

This code is tested under Ubuntu 18.04, CUDA 11.2, with one NVIDIA Titan RTX GPU. Python 3.8.8 version is used for development.

Datasets

• Weizmann Horse Database, Download here
• CMU Mocap Download here

Results

Binary Segmentation

Set 'data_dir' and 'run_dir' in 'experiments/img_seg/flags_img_seg.py'
Run 'experiments/img_seg/train.py'

CMU Mocap

Prepare the dataset by running 'experiments/motion/preprocess_datasets.py'
Set 'data_dir' and 'run_dir' in 'experiments/motion/flags_motion.py'
For training, run 'experiments/motion/train.py'
For evaluation and visualization, run 'experiments/motion/evaluation.py'

Citation

Please cite our paper if you use code from this repository:

@ARTICLE{10186219,
  author={Zand, Mohsen and Etemad, Ali and Greenspan, Michael},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence}, 
  title={Flow-Based Spatio-Temporal Structured Prediction of Motion Dynamics}, 
  year={2023},
  volume={45},
  number={11},
  pages={13523-13535},
  doi={10.1109/TPAMI.2023.3296446}}

References

Conditional-Glow
Locally Masked Convolution