PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction System

Huayi Wang*, Wentao Zhang*, Runyi Yu*, Tao Huang, Junli Ren, Feiyu Jia, Zirui Wang,
Xiaojie Niu, Xiao Chen, Jiahe Chen, Qifeng Chen^†, Jingbo Wang^†, Jiangmiao Pang^†
*Equal Contributions    ^†Corresponding Authors

🛠️ Installation

1. Create a conda environment and install PyTorch:

conda create -n physhsi python=3.8
conda activate physhsi
pip3 install torch torchvision --index-url https://download.pytorch.org/whl/cu118

2. Install Isaac Gym:

• Download and install Isaac Gym Preview 4 from NVIDIA Developer.
• Navigate to its Python folder and install.

  cd isaacgym/python && pip install -e .

3. Clone this repository.

git clone https://github.com/InternRobotics/PhysHSI.git
cd PhysHSI

4. Install PhysHSI.

cd rsl_rl && pip install -e .
cd ../legged_gym && pip install -e .

5. Install additional dependencies.

cd .. && pip install -r requirements.txt

🕹️ Run PhysHSI

PhysHSI supports six tasks for the Unitree G1 humanoid robot: CarryBox, SitDown, LieDown, StandUp, StyleLoco-Dinosaur, and StyleLoco-Highknee.

Motion Visualization

Reference motion data for each task can be found in the motion data folder. To visualize reference motion data, run:

cd legged_gym
python legged_gym/scripts/play.py --task [task_name] --play_dataset

Here, [task_name] can be one of [carrybox, liedown, sitdown, standup, styleloco_dinosaur, styleloco_highknee].

Play with Pre-trained Checkpoints

Pre-trained checkpoints for each task are available in the checkpoint folder. To play a task using a checkpoint, run:

python legged_gym/scripts/play.py --task [task_name] --resume_path resources/ckpt/[task_name].pt

For example, to play the CarryBox task:

python legged_gym/scripts/play.py --task carrybox --resume_path resources/ckpt/carrybox.pt

⚠️ Note:

During the first 1–2 episodes of play.py, you may observe slight interpenetration between the robot, the object, or the platform.

This issue only occurs in the initial episodes and does not affect training or subsequent performance.

🤖 Train PhysHSI

CarryBox

CarryBox is a challenging long-horizon task. We train it in two steps:

1. Initial training: Use a relatively small AMP coefficient and relaxed termination conditions for easier learning. Run approximately 20k steps:

   python legged_gym/scripts/train.py --task carrybox --headless 

2. Refined training: To better align with the data, manually increase the AMP coefficient and continue training for about 30~40k steps:

   python legged_gym/scripts/train.py --task carrybox_resume --resume --resume_path [ckpt_path] --headless

   Here, [ckpt_path] refers to the checkpoint from the first 20k-step training stage.

To play the final trained checkpoint:

python legged_gym/scripts/play.py --task carrybox --resume_path [ckpt_path]

Other Tasks

For the remaining five tasks, you can directly train them using:

python legged_gym/scripts/train.py --task [task_name] --headless

Here, [task_name] can be one of [liedown, sitdown, standup, styleloco_dinosaur, styleloco_highknee].

To play the final trained checkpoint for any task:

python legged_gym/scripts/play.py --task [task_name] --resume_path [ckpt_path]

By default, PhysHSI uses TensorBoard for logging training metrics.

If you prefer to use Weights & Biases (wandb), please enable it in the corresponding [task_name]_config.py file and set the appropriate wandb_entity for your account.

👏 Acknowledgements

This repository is built upon the support and contributions of the following open-source projects. Special thanks to:

• Legged_gym and HIMLoco: The foundation environments for training and running codes.
• RSL_RL: Reinforcement learning algorithm implementation.
• AMP for Hardware and TokenHSI: References for AMP and RSI implementations.
• AMASS, SAMP and 100STYLE: Reference dataset construction.

🔗 Citation

If you find our work helpful, please cite:

@article{wang2025physhsi,
  title   = {PhysHSI: Towards a Real-World Generalizable and Natural Humanoid-Scene Interaction System},
  author  = {Wang, Huayi and Zhang, Wentao and Yu, Runyi and Huang, Tao and Ren, Junli and Jia, Feiyu and Wang, Zirui and Niu, Xiaojie and Chen, Xiao and Chen, Jiahe and Chen, Qifeng and Wang, Jingbo and Pang, Jiangmiao},
  journal = {arXiv preprint arXiv:2510.11072},
  year    = {2025},
}

📄 License

The PhysHSI code is licensed under the CC BY-NC-SA 4.0 International License . Commercial use is not allowed without explicit authorization.