Fork Modifications

Original Repository: unitreerobotics/unitree_rl_lab 🍴 This Fork: mintlabkorea/unitree_g1_rl_lab

This fork extends the original Unitree RL Lab with advanced symmetry-based policy architectures and in-place yaw turning capabilities for more robust real-world robot control.

Important Notes

1. Symmetric training strategy is only conceptual and has not been trained or validated yet.
2. finetune_yaw_turning_v2.py is the version used to train, validate in sim2sim, sim2real.
3. Robot succeeded to make in-place turns in mujoco.
4. Real robot failed, but moved its legs to shift its weights. (No such movement in the original policy)

Key Enhancements

1. Multi-Symmetry Policy Architectures

• Mirrored Policies: Left-right symmetry for balanced locomotion
• Symmetric Policies: Full body symmetry optimization
• Multi-Symmetric Policies: Combined symmetry approaches for enhanced robustness
• Max-Symmetric Policies: Maximum symmetry enforcement

2. Advanced Yaw Turning Capability

• In-place Yaw Turning: Fine-tuned policies for rotational movement while preserving walking performance
• Conservative Fine-tuning: Gradual parameter modification to maintain existing locomotion quality
• Comprehensive Validation: Safety protocols and performance metrics for real-world deployment

3. Enhanced Training Scripts

• Parallel Training Architectures: Multiple symmetry training approaches
• Fine-tuning Pipelines: Specialized scripts for policy enhancement
• Validation Tools: Comprehensive testing and safety validation systems

New Files and Directories

Training Scripts

scripts/
├── finetune_yaw_turning.py           # Main yaw turning fine-tuning script
├── rsl_rl/
│   ├── train_mirrored.py            # Mirrored symmetry training
│   ├── train_multisymmetric.py       # Multi-symmetry architecture training
│   ├── train_multisymmetric_simple.py # Simplified multi-symmetry approach
│   ├── train_symmetric.py            # Basic symmetry training
│   └── finetune_yaw_turning_v2.py    # Advanced yaw fine-tuning
├── test_multisymmetric_policy.py     # Policy testing and validation
└── validate_yaw_policy.py            # Yaw turning safety validation

Environment Configurations

source/unitree_rl_lab/tasks/locomotion/robots/g1/29dof/
├── velocity_env_cfg_finetune.py      # Fine-tuning environment config
├── velocity_env_cfg_mirrored.py      # Mirrored policy environment
├── velocity_env_cfg_multisymmetric.py # Multi-symmetry environment
├── velocity_env_cfg_symmetric.py     # Symmetric policy environment
└── velocity_env_cfg_maxsymmetric.py  # Maximum symmetry environment

Utility Systems

source/unitree_rl_lab/utils/
├── multisymmetric_policy.py          # Multi-symmetry policy implementation
├── symmetric_policy.py               # Basic symmetry policy wrapper
├── maxsymmetric_policy.py            # Maximum symmetry enforcement
└── deploy_mirror.py                  # Deployment mirroring utilities

Agent Configurations

source/unitree_rl_lab/tasks/locomotion/agents/
└── rsl_rl_ppo_cfg_finetune.py        # Fine-tuning agent configuration

Documentation

├── YAWTRAINING_PARAMETER_CHANGES.txt # Detailed parameter tracking log
├── YAW_TURNING_DEPLOYMENT_GUIDE.md   # Complete deployment guide
└── docs/                             # Additional documentation

Model Assets

deploy/robots/g1_29dof/config/policy/yaw/ # Yaw turning policy configurations
unitree_model/                            # Robot model assets
IsaacLab/                                 # IsaacLab integration

Core Modifications Summary

Modified Core Files

1. velocity_env_cfg.py: Enhanced with symmetry support and yaw turning parameters
2. __init__.py: Extended task registration for new environment variants
3. rsl_rl_ppo_cfg.py: PPO configuration enhancements for stable fine-tuning
4. unitree.py: Robot asset configuration updates
5. config.yaml: Deployment configuration adjustments

Parameter Optimizations

• Reward Structure: Balanced linear and angular velocity tracking (1.0:0.7 ratio)
• Command Ranges: Conservative expansion for safe yaw turning (±0.4 rad/s)
• Training Parameters: Reduced learning rates for stable fine-tuning (5.0e-4)
• Episode Length: Extended to 25 seconds for complex scenarios

Safety Enhancements

• Conservative Fine-tuning: Preserves existing walking performance
• Validation Protocols: Comprehensive safety testing before real robot deployment
• Progressive Testing: Gradual speed increase schedules
• Robustness Metrics: >70% recovery rate from disturbances

Quick Start with New Features

Train with Multi-Symmetry

python scripts/rsl_rl/train_multisymmetric_simple.py \
    --task Isaac-Velocity-G1-29dof-MultiSymmetric-v0 \
    --num_envs 4096 \
    --max_iterations 100

Fine-tune for Yaw Turning

python scripts/finetune_yaw_turning.py \
    --load_run "your_base_model_run" \
    --checkpoint "model_50000.pt" \
    --max_iterations 5000

Validate Before Deployment

python scripts/validate_yaw_policy.py \
    --checkpoint "path/to/finetuned/model.pt" \
    --save_videos --save_plots

Technical Architecture

Symmetry Framework

• Mirror Symmetry: Enforces left-right balance for stable locomotion
• Multi-Level Symmetry: Combines joint-level and action-level symmetry constraints
• Adaptive Symmetry: Dynamic symmetry enforcement based on task requirements

Fine-tuning Pipeline

• Conservative Learning: Reduced learning rates to prevent catastrophic forgetting
• Reward Balancing: Maintains walking quality while adding rotational capability
• Parameter Tracking: Detailed logs of all training modifications

Deployment Safety

• Simulation Validation: Comprehensive testing in Isaac Lab before real deployment
• Progressive Rollout: Step-by-step performance validation protocol
• Emergency Protocols: Safety stops and recovery procedures

Unitree RL Lab

Overview

This project provides a set of reinforcement learning environments for Unitree robots, built on top of IsaacLab.

Currently supports Unitree Go2, H1 and G1-29dof robots.

Isaac Lab	Mujoco	Physical

Installation

• Install Isaac Lab by following the installation guide.

• Install the Unitree RL IsaacLab standalone environments.

  • Clone or copy this repository separately from the Isaac Lab installation (i.e. outside the IsaacLab directory):

    git clone https://github.com/unitreerobotics/unitree_rl_lab.git

  • Use a python interpreter that has Isaac Lab installed, install the library in editable mode using:

    conda activate env_isaaclab
    python -m pip install -e source/unitree_rl_lab

• Download unitree usd files

  • Download unitree usd files from unitree_model, keeping folder structure

    git lfs install
    git clone https://huggingface.co/datasets/unitreerobotics/unitree_model

  • Config UNITREE_MODEL_DIR in source/unitree_rl_lab/unitree_rl_lab/assets/robots/unitree.py.

    UNITREE_MODEL_DIR = "</home/user/projects/unitree_usd>"

• Verify that the environments are correctly installed by:

  • Listing the available tasks:

    python scripts/list_envs.py

  • Running a task:

    python scripts/rsl_rl/train.py --headless --task Unitree-G1-29dof-Velocity

  • Inference with a trained agent:

    python scripts/rsl_rl/play.py --task Unitree-G1-29dof-Velocity

Deploy

After the model training is completed, we need to perform sim2sim on the trained strategy in Mujoco to test the performance of the model. Then deploy sim2real.

Setup

# Install dependencies
sudo apt install -y libyaml-cpp-dev libboost-all-dev libeigen3-dev libspdlog-dev libfmt-dev
# Install unitree_sdk2
git clone git@github.com:unitreerobotics/unitree_sdk2.git
cd unitree_sdk2
mkdir build && cd build
cmake .. -DBUILD_EXAMPLES=OFF # Install on the /usr/local directory
sudo make install
# Compile the robot_controller
cd unitree_rl_lab/deploy/robots/g1_29dof # or other robots
mkdir build && cd build
cmake .. && make

Sim2Sim

Installing the unitree_mujoco.

• Set the robot at /simulate/config.yaml to g1
• Set domain_id to 0
• Set enable_elastic_hand to 1
• Set use_joystck to 1.

# start simulation
cd unitree_mujoco/simulate/build
./unitree_mujoco
# ./unitree_mujoco -i 0 -n eth0 -r g1 -s scene_29dof.xml # alternative

cd unitree_rl_lab/deploy/robots/g1_29dof/build
./g1_ctrl
# 1. press [L2 + Up] to set the robot to stand up
# 2. Click the mujoco window, and then press 8 to make the robot feet touch the ground.
# 3. Press [R1 + X] to run the policy.
# 4. Click the mujoco window, and then press 9 to disable the elastic band.

Sim2Real

You can use this program to control the robot directly, but make sure the on-borad control program has been closed.

./g1_ctrl --network eth0 # eth0 is the network interface name.

---

Acknowledgements

This repository is built upon the support and contributions of the following open-source projects. Special thanks to:

• IsaacLab: The foundation for training and running codes.
• mujoco: Providing powerful simulation functionalities.
• robot_lab: Referenced for project structure and parts of the implementation.