DiffMimic:
Efficient Motion Mimicking with Differentiable Physics

Jiawei Ren^* Cunjun Yu^* Siwei Chen Xiao Ma Liang Pan Ziwei Liu^†

S-Lab, Nanyang Technological University  National University of Singapore  
^*equal contribution
^†corresponding author

ICLR 2023

---

[Project Page] • [Paper] • [Demo] • [Video]

About

We implement DiffMimic with Brax:

Brax is a fast and fully differentiable physics engine used for research and development of robotics, human perception, materials science, reinforcement learning, and other simulation-heavy applications.

An environment mimic_env is implemented for training and benchmarking. mimic_env now includes the following characters:

• HUMANOID: AMP-formatted humanoid, used for acrobatics skills.
• SMPL: SMPL-formatted humanoid, used for mocap data.
• SWORDSHIELD: ASE-formatted humanoid, used for REALLUSION sword-shield motion.

More characters are on the way.

Installation

conda create -n diffmimic python==3.9 -y
conda activate diffmimic

pip install --upgrade pip
pip install --upgrade "jax[cuda]==0.4.2" -f https://storage.googleapis.com/jax-releases/jax_cuda_releases.html
pip install -r requirements.txt

Get Started

python mimic.py --config configs/AMP/backflip.yaml

Visualize

streamlit run visualize.py

Citation

If you find our work useful for your research, please consider citing the paper:

@inproceedings{ren2023diffmimic,
  author    = {Ren, Jiawei and Yu, Cunjun and Chen, Siwei and Ma, Xiao and Pan, Liang and Liu, Ziwei},
  title     = {DiffMimic: Efficient Motion Mimicking with Differentiable Physics},
  journal   = {ICLR},
  year      = {2023},
}

Acknowledgment

• Differentiable physics simulation is done by Brax.
• Early version of the code is heavily based on Imitation Learning via Differentiable Physics (ILD).
• Motion files are borrowed from DeepMimic, ASE, AMASS, and AIST++.
• Characters are borrowed from DeepMimic and ASE.
• The work is inspired by valuable insights from SuperTrack and Spacetime Bound.