RL_Dog

Reinforced-Learning for autonomous walking and suddenly-stopping of Legged Robot (AlienGO by Unitree)

Project by Pietro Dardano, advised by prof. A. Del Prete - UniTn - Summer 2024

Methodology

• Proximal Policy Optimization (PPO) and TODO: Constraints as Terminations (CAT): For detailed information on these methods, refer to the research paper.
• Architecture Inspired by ANYmal (ETH-RSL): We modeled our architecture based on the principles outlined in the ANYmal paper.
• SKRL: We utilized the SKRL library to streamline our reinforcement learning implementations. More details can be found in the SKRL documentation.
• Python + PyTorch: Our primary programming languages and framework for development and deep learning.

Setup

Workstation

• NVIDIA's Isaac Lab: provides the high-performance simulation environment necessary for training our models. Refer to the Orbit and Isaac Sim pages for more information.
  

Config:

• CPU: AMD® Ryzen 9 7950x
• GPU: 2x NVIDIA RTX A6000 Ada Generation, 48Gb GDDR6, 300W
• RAM: 192Gb
• OS: Ubuntu 22.04.4 LTS

Please note that IsaacLab contains many OpenAI Gym and Gymnasium features. It is common to find attributes, methods and classes related to them.
It contains RSL_RL too, helpfull framework by ETH for legged robot training.

Laptop (lower grade simulations)

Remark: for the time being i am mainly working on the Isaac Sim+Lab version for a more complete and realistic simulation. I'll try my best to implement it soon.

• OpenAI Gymnasium: Since Isaac Sim is almost not suitable for being installed on laptops, I opted for the lightweight Gymnasium as a simulation environment. It supports Python 3.10 and Ubuntu 22.04 and it's well documented. Obviously, it is far from a realistic simulation, as Isaac Sim is, but for quick tests and trainings, I consider it a good trade-off considering my hardware limitations. For more details on Gymnasium, visit the official documentation.

Why not Isaac Gym?:

It requires Ubuntu 20.04 or earlier and Python 3.8 or 3.9. Having installed Ubuntu 22.04, I excluded this option.
It is deprecated too, everyone now-a-day is transitioning to IsaacLab

Understanding the Project

For a comprehensive understanding of the principles and techniques used in this project, refer to the following resources:

• A detailed review of related methodologies can be found in reference 1.
• Insights into recent advancements are discussed in reference 2.

Project Structure

=== TBD ===

Installation

To set up the project, follow these steps:

1. Setup your OS and Environment Instructions in the file: IsaacSim-Setup_Guide or TODO_Gymnasium-Setup_Guide
2. Clone the repository:

   git clone https://github.com/PietroDrd/RL_Dog.git