GMR: General Motion Retargeting

Installation

Note

The code is tested on Ubuntu 22.04/20.04.

First create your conda environment:

conda create -n gmr python=3.10 -y
conda activate gmr

Then, install GMR:

pip install -e .

After installing SMPLX, change ext in smplx/body_models.py from npz to pkl if you are using SMPL-X pkl files.

And to resolve some possible rendering issues:

conda install -c conda-forge libstdcxx-ng -y

Data Preparation

[SMPLX body model] download SMPL-X body models to assets/body_models from SMPL-X and then structure as follows:

- assets/body_models/smplx/
-- SMPLX_NEUTRAL.pkl
-- SMPLX_FEMALE.pkl
-- SMPLX_MALE.pkl

[AMASS motion data] download raw SMPL-X data to any folder you want from AMASS. NOTE: Do not download SMPL+H data.

[OMOMO motion data] download raw OMOMO data to any folder you want from this google drive file. And process the data into the SMPL-X format using scripts/convert_omomo_to_smplx.py.

[LAFAN1 motion data] download raw LAFAN1 bvh files from the official repo, i.e., lafan1.zip.

Human/Robot Motion Data Formulation

To better use this library, you can first have an understanding of the human motion data we use and the robot motion data we obtain.

Each frame of human motion data is formulated as a dict of (human_body_name, 3d global translation + global rotation). The rotation is usually represented as quaternion (with wxyz order by default, to align with mujoco).

Each frame of robot motion data can be understood as a tuple of (robot_base_translation, robot_base_rotation, robot_joint_positions).

Usage

[NEW] PICO Streaming to Robot (TWIST2)

Install PICO SDK:

1. On your PICO, install PICO SDK: see here.
2. On your own PC,
   • Download deb package for ubuntu 22.04, or build from the repo source.
   • To install, use command

     sudo dpkg -i XRoboToolkit_PC_Service_1.0.0_ubuntu_22.04_amd64.deb

     then you should see xrobotoolkit-pc-service in your APPs. remember to start this app before you do teleopperation.
   • Build PICO PC Service SDK and Python SDK for PICO streaming:

     conda activate gmr
     
     git clone https://github.com/YanjieZe/XRoboToolkit-PC-Service-Pybind.git
     cd XRoboToolkit-PC-Service-Pybind
     
     mkdir -p tmp
     cd tmp
     git clone https://github.com/XR-Robotics/XRoboToolkit-PC-Service.git
     cd XRoboToolkit-PC-Service/RoboticsService/PXREARobotSDK 
     bash build.sh
     cd ../../../..
     
     
     mkdir -p lib
     mkdir -p include
     cp tmp/XRoboToolkit-PC-Service/RoboticsService/PXREARobotSDK/PXREARobotSDK.h include/
     cp -r tmp/XRoboToolkit-PC-Service/RoboticsService/PXREARobotSDK/nlohmann include/nlohmann/
     cp tmp/XRoboToolkit-PC-Service/RoboticsService/PXREARobotSDK/build/libPXREARobotSDK.so lib/
     # rm -rf tmp
     
     # Build the project
     conda install -c conda-forge pybind11
     pip uninstall -y xrobotoolkit_sdk
     python setup.py install

You should be all set!

To try it, check this script from TWIST2:

bash teleop.sh

Visualize saved robot motion

Visualize a single motions:

python scripts/vis_robot_motion.py --robot <robot_name> --robot_motion_path <path_to_save_robot_data.pkl>

If you want to record video, add --record_video and --video_path <your_video_path,mp4>.

Visualize a folder of motions:

python scripts/vis_robot_motion_dataset.py --robot <robot_name> --robot_motion_folder <path_to_save_robot_data_folder>

After launching the MuJoCo visualization window and clicking on it, you can use the following keyboard controls::

• [: play the previous motion
• ]: play the next motion
• space: toggle play/pause

Speed Benchmark

CPU	Retargeting Speed
AMD Ryzen Threadripper 7960X 24-Cores	60~70 FPS
13th Gen Intel Core i9-13900K 24-Cores	35~45 FPS
TBD	TBD

Citation

@article{joao2025gmr,
  title={Retargeting Matters: General Motion Retargeting for Humanoid Motion Tracking},
  author= {Joao Pedro Araujo and Yanjie Ze and Pei Xu and Jiajun Wu and C. Karen Liu},
  year= {2025},
  journal= {arXiv preprint arXiv:2510.02252}
}

@article{ze2025twist,
  title={TWIST: Teleoperated Whole-Body Imitation System},
  author= {Yanjie Ze and Zixuan Chen and João Pedro Araújo and Zi-ang Cao and Xue Bin Peng and Jiajun Wu and C. Karen Liu},
  year= {2025},
  journal= {arXiv preprint arXiv:2505.02833}
}

and this github repo:

@software{ze2025gmr,
  title={GMR: General Motion Retargeting},
  author= {Yanjie Ze and João Pedro Araújo and Jiajun Wu and C. Karen Liu},
  year= {2025},
  url= {https://github.com/YanjieZe/GMR},
  note= {GitHub repository}
}

Acknowledgement

This IK solver is built upon mink and mujoco. Our visualization is built upon mujoco. The human motion data we try includes AMASS, OMOMO, and LAFAN1.