Awesome-Motion-Diffusion-Models

A collection of resources and papers on Motion Diffusion Models.

The template and some of the components are referenced from Awesome-Diffusion-Model.

Contents

• Datasets

  • Text to Motion
    • HumanML3D
    • KIT-ML
    • BABEL
  • Audio to Motion
    • AIST++
  • Motion Prediction
    • Human3.6M
    • AMASS
    • HumanEva

• Papers

  • Survey
  • Text to Motion
  • Audio to Motion
  • Speech to Motion
  • Text & Audio to Motion
  • Motion Prediction

Datasets

Text to Motion

HumanML3D

Generating Diverse and Natural 3D Human Motions from Text
Chuan Guo, Shihao Zou, Xinxin Zuo, Sen Wang, Wei Ji, Xingyu Li, Li Cheng
CVPR 2022. [Paper] [Project] [Code]

KIT-ML

The KIT Motion-Language Dataset
Matthias Plappert, Christian Mandery, Tamim Asfour
Big Data 2016. [Paper] [Project]

BABEL

BABEL: Bodies, Action and Behavior with English Labels
Abhinanda R. Punnakkal, Arjun Chandrasekaran, Nikos Athanasiou, Alejandra Quiros-Ramirez, Michael J. Black
CVPR 2021. [Paper] [Project] [Code]

Audio to Motion

AIST++

AI Choreographer: Music Conditioned 3D Dance Generation with AIST++
Ruilong Li, Shan Yang, David A. Ross, Angjoo Kanazawa
ICCV 2021. [Paper] [Project] [Code]

Motion Prediction

Human3.6M

Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments
Catalin Ionescu, Dragos Papava, Vlad Olaru, Cristian Sminchisescu
TPAMI 2014. [Paper] [Project]

AMASS

AMASS: Archive of Motion Capture As Surface Shapes
Naureen Mahmood, Nima Ghorbani, Nikolaus F. Troje, Gerard Pons-Moll, Michael J. Black
ICCV 2019. [Paper] [Project] [Code]

HumanEva

HUMANEVA: Synchronized Video and Motion Capture Dataset and Baseline Algorithm for Evaluation of Articulated Human Motion IJCV 2010. [Paper] [Project]

Papers

Survey

Text to Motion

Human Motion Diffusion Model
Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Amit H. Bermano, Daniel Cohen-Or
arXiv 2022. [Paper] [Project] [Code]

SinMDM: Single Motion Diffusion
Anonymous Authors
[Paper] [Project] [Code]

PhysDiff: Physics-Guided Human Motion Diffusion Model
Ye Yuan, Jiaming Song, Umar Iqbal, Arash Vahdat, Jan Kautz
arXiv 2022. [Paper] [Project]

MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Model
Mingyuan Zhang, Zhongang Cai, Liang Pan, Fangzhou Hong, Xinying Guo, Lei Yang, Ziwei Liu
arXiv 2022. [Paper] [Project] [Code]

FLAME: Free-form Language-based Motion Synthesis & Editing
Jihoon Kim, Jiseob Kim, Sungjoon Choi
arXiv 2022. [Paper]

Diffusion Motion: Generate Text-Guided 3D Human Motion by Diffusion Model
Zhiyuan Ren, Zhihong Pan, Xin Zhou, Le Kang
arXiv 2022. [Paper]

Executing your Commands via Motion Diffusion in Latent Space
Xin Chen, Biao Jiang, Wen Liu, Zilong Huang, Bin Fu,Tao Chen, Jingyi Yu, Gang Yu
CVPR 2023. [Paper] [Project] [Code]

InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactions
Han Liang, Wenqian Zhang, Wenxuan Li, Jingyi Yu, Lan Xu
arXiv 2023. [Paper]

Human Motion Diffusion as a Generative Prior
Yonatan Shafir, Guy Tevet, Roy Kapon, Amit H. Bermano
arXiv 2023. [Paper] [Project] [Code]

Text2Performer: Text-Driven Human Video Generation
Yuming Jiang, Shuai Yang, Tong Liang Koh, Wayne Wu, Chen Change Loy, Ziwei Liu
arXiv 2023. [Paper] [Project] [Code]

ReMoDiffuse: Retrieval-Augmented Motion Diffusion Model
Mingyuan Zhang, Xinying Guo, Liang Pan, Zhongang Cai, Fangzhou Hong, Huirong Li, Lei Yang, Ziwei Liu
arXiv 2023. [Paper] [Project] [Github]

Audio to Motion

Listen, denoise, action! Audio-driven motion synthesis with diffusion models
Simon Alexanderson, Rajmund Nagy, Jonas Beskow, Gustav Eje Henter
arXiv 2022. [Paper] [Project]

EDGE: Editable Dance Generation From Music
Jonathan Tseng, Rodrigo Castellon, C. Karen Liu
CVPR 2023. [Paper] ][Project] [Code]

Speech to Motion

DiffMotion: Speech-Driven Gesture Synthesis Using Denoising Diffusion Model
Fan Zhang, Naye Ji, Fuxing Gao, Yongping Li
MMM 2023. [Paper] [Project] [Code]

Text & Audio to Motion

UDE: A Unified Driving Engine for Human Motion Generation
Zixiang Zhou, Baoyuan Wang
CVPR 2023. [Paper] [Project] [Code]

MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesis
Rishabh Dabral, Muhammad Hamza Mughal, Vladislav Golyanik, Christian Theobalt
CVPR 2023. [Paper] [Project]

Motion Prediction

BeLFusion: Latent Diffusion for Behavior-Driven Human Motion Prediction
German Barquero, Sergio Escalera, Cristina Palmero
arXiv 2022. [Paper] [Project] [Code]

HumanMAC: Masked Motion Completion for Human Motion Prediction
Ling-Hao Chen, Jiawei Zhang, Yewen Li, Yiren Pang, Xiaobo Xia, Tongliang Liu
arXiv 2023. [Paper] [Project] [Code]

Avatars Grow Legs: Generating Smooth Human Motion from Sparse Tracking Inputs with Diffusion Model
Yuming Du, Robin Kips, Albert Pumarola, Sebastian Starke, Ali Thabet, Artsiom Sanakoyeu
CVPR 2023. [Paper] [Project] [Code]