Model-Based RSL RL

A fast and simple implementation of RL algorithms, designed to run fully on GPU. This code is a fork of RSL RL incorporated with model-based RL algorithms supporting Robotic World Model.

Paper: Robotic World Model: A Neural Network Simulator for Robust Policy Optimization in Robotics
Project Page: https://sites.google.com/view/roboticworldmodel

Authors: Chenhao Li, Andreas Krause, Marco Hutter
Affiliation: ETH AI Center, Learning & Adaptive Systems Group and Robotic Systems Lab, ETH Zurich

Setup

The package can be installed via PyPI with:

pip install rsl-rl-lib

or by cloning this repository and installing it with:

git clone https://github.com/leggedrobotics/rsl_rl
cd rsl_rl
pip install -e .

The package supports the following logging frameworks which can be configured through logger:

• Tensorboard: https://www.tensorflow.org/tensorboard/
• Weights & Biases: https://wandb.ai/site
• Neptune: https://docs.neptune.ai/

For a demo configuration of PPO, please check the example_config.yaml file.

Citing

If you use the library with model-based reinforcement learning, please cite the following work:

@article{li2025robotic,
  title={Robotic world model: A neural network simulator for robust policy optimization in robotics},
  author={Li, Chenhao and Krause, Andreas and Hutter, Marco},
  journal={arXiv preprint arXiv:2501.10100},
  year={2025}
}