Reinforced-Learning for autonomous walking and suddenly-stopping of Legged Robot (AlienGO by Unitree)
Project by Pietro Dardano, advised by prof. A. Del Prete - UniTn - Summer 2024
- Proximal Policy Optimization (PPO) and TODO: Constraints as Terminations (CAT): For detailed information on these methods, refer to the research paper.
- Architecture Inspired by ANYmal (ETH-RSL): We modeled our architecture based on the principles outlined in the ANYmal paper.
- SKRL: We utilized the SKRL library to streamline our reinforcement learning implementations. More details can be found in the SKRL documentation.
- Python + PyTorch: Our primary programming languages and framework for development and deep learning.
- NVIDIA's Isaac Lab: provides the high-performance simulation environment necessary for training our models. Refer to the Orbit and Isaac Sim pages for more information.
- CPU: AMD® Ryzen 9 7950x
- GPU: 2x NVIDIA RTX A6000 Ada Generation, 48Gb GDDR6, 300W
- RAM: 192Gb
- OS: Ubuntu 22.04.4 LTS
Please note that IsaacLab contains many OpenAI Gym and Gymnasium features. It is common to find attributes, methods and classes related to them.
It contains RSL_RL too, helpfull framework by ETH for legged robot training.
Remark: for the time being i am mainly working on the Isaac Sim+Lab version for a more complete and realistic simulation. I'll try my best to implement it soon.
- OpenAI Gymnasium: Since Isaac Sim is almost not suitable for being installed on laptops, I opted for the lightweight Gymnasium as a simulation environment. It supports Python 3.10 and Ubuntu 22.04 and it's well documented. Obviously, it is far from a realistic simulation, as Isaac Sim is, but for quick tests and trainings, I consider it a good trade-off considering my hardware limitations. For more details on Gymnasium, visit the official documentation.
It requires Ubuntu 20.04 or earlier and Python 3.8 or 3.9. Having installed Ubuntu 22.04, I excluded this option.
It is deprecated too, everyone now-a-day is transitioning to IsaacLab
For a comprehensive understanding of the principles and techniques used in this project, refer to the following resources:
- A detailed review of related methodologies can be found in reference 1.
- Insights into recent advancements are discussed in reference 2.
=== TBD ===
To set up the project, follow these steps:
- Setup your OS and Environment Instructions in the file: IsaacSim-Setup_Guide or TODO_Gymnasium-Setup_Guide
- Clone the repository:
git clone https://github.com/PietroDrd/RL_Dog.git