Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations

Zhixue Fang^*,1, Zhimin Zhang^*,1,2, Bi'an Du^1,2, Zijie Meng², Yan Zhou^†,1, Wei Hu^†,2, Guoxin Zhang¹, Pengfei Wan¹, Kun Gai¹

(*Equal contribution, †Corresponding author)

¹ Kling Team    ² Peking University

---

TL;DR: We enable open-category motion transfer beyond fixed structural correspondence through a two-stage framework that learns transferable motion abstractions and internalizes them into direct reference-video-conditioned generation, achieving state-of-the-art motion fidelity and target preservation across large morphological gaps.

Cross-Category Motion Transfer

1.2.mp4

2.1.mp4

Overview

Overview of our two-stage framework. Left: Stage I learns heterogeneous abstract motion conditions and bootstraps cross-category motion pairs. Upper right: Stage II internalizes this supervision with raw reference videos. Lower right: Inference directly conditions on a reference video, text, and an optional reference image.

🌟Citation

Please give us a star 🌟 and cite our paper if you find our work helpful.

@misc{fang2026motionmorphologybootstrappingcrosscategory,
      title={Motion Beyond Morphology: Bootstrapping Cross-Category Motion Transfer from Abstract Motion Representations}, 
      author={Zhixue Fang and Zhimin Zhang and Bi'an Du and Zijie Meng and Yan Zhou and Wei Hu and Guoxin Zhang and Pengfei Wan and Kun Gai},
      year={2026},
      eprint={2608.01628},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2608.01628}, 
}