How does Skrl compare to Gymnasium, Stable Baselines3, TF-Agents, etc.

[AmeerArsala]

Beginner here. I'd like to know how Skrl compares to the other RL libraries and what makes it unique. I know it's built on top of PyTorch, and provides a neat bridge between PyTorch and Gymnasium, but is that all? Does that mean that TF-Agents is basically Skrl, but for TensorFlow? Since it is also built on top of JAX, would it also provide a bridge to JAX/Flax, or would it be either/or? Since JAX is really fast, when using it, is that speed incorporated into the PyTorch components, or is it mutually exclusive?

[Toni-SM]

Hi @AmeerArsala

• Some points of comparison of skrl with other libraries can be analyzed through the main skrl features:

  • End-to-end implementation in PyTorch and JAX
  • Clear and readable code
  • Modularity and reusability
  • Documented library, code and implementations
  • Single and multi-agent (the last one, with few agents at the moment but in progress)
  • Support for different environment interfaces:
    • Gym/Gymnasium (single and vectorized),
    • Shimmy,
    • DeepMind,
    • robosuite,
    • PettingZoo,
    • Bi-DexHands,
    • NVIDIA Isaac Gym (preview 2, 3 and 4),
    • NVIDIA Isaac Orbit,
    • NVIDIA Omniverse Isaac Gym
  • Simultaneous learning by scopes in Gym/Gymnasium (vectorized), NVIDIA Isaac Gym, NVIDIA Isaac Orbit and NVIDIA Omniverse Isaac Gym.

• Regarding TF-Agents:

  From a generic view, TF-Agents is like the Tensorflow version of skrl. However, skrl has a more modular design, a more complete documentation and allows greater access to different environment interfaces, such as NVIDIA Isaac family, for example :)

• Regarding PyTorch and JAX (both ML frameworks):

  The skrl components (agents, environment wrappers, modes, etc) are implemented using both frameworks. You can use one of both, the PyTorch implementation or the JAX implementation. Currently there is no way to combine them.

  In the case of JAX, the implementation also uses Flax (for the models: artificial neural networks) and Optax (for optimizers and learning rate schedulers).