Benchmark results

[Toni-SM]

This topic presents the benchmark results of the skrl library in different environments and with different algorithms

These benchmarks are a work in progress that will be updated (editing or adding new results) over time.
The content will remain locked. Open a new discussion to debate them!

---

Content (may be in a different order):

• NVIDIA Isaac Gym
• NVIDIA Isaac Orbit
• NVIDIA Omniverse Isaac Gym
• Gym / Gymnasium

[Toni-SM]

NVIDIA Isaac Gym

---

Benchmark

Setup

Package	Version
Isaac Gym	preview 4
IsaacGymEnvs	commit ca7a4fb762f9581e39cc2aab644f18a83d6ab0ba

Results

Environment	PPO
AllegroHand 1	3942.69
Ant	5466.3 +/- 279.61
Anymal	61.86 +/- 1.81
AnymalTerrain	19.82 +/- 0.57
BallBalance	288.07 +/- 25.54
Cartpole 2	494.34 +/- 0.87
FrankaCabinet	3134.0 +/- 182.99
Humanoid	6474.34 +/- 696.27
Ingenuity	7066.82 +/- 488.97
Quadcopter	1237.75 +/- 127.05
ShadowHand	7898.38 +/- 180.75

Environment	AMP
Humanoid	295.65 +/- 0.86

¹ one run
² variable behavior to the random number generator seed

Comparison (single run)

• rl_games-v1.5.2
  • AllegroHand: PPO (3856)
  • Ant: PPO (6470)
  • Anymal: PPO (62)
  • AnymalTerrain: PPO (19)
  • BallBalance: PPO (291)
  • Cartpole: PPO (492)
  • FrankaCabinet: PPO (3317)
  • Humanoid: PPO (8952)
  • Ingenuity: PPO (6296)
  • Quadcopter: PPO (1326)
  • ShadowHand: PPO (7906)

Visualizations

The following charts show the episode's mean length in timesteps (left) and the mean total reward (right)

AllegroHand (PPO)

Ant (PPO)

Anymal (PPO)

AnymalTerrain (PPO)

BallBalance (PPO)

Cartpole (PPO)

FrankaCabinet (PPO)

Humanoid (PPO)

Humanoid (AMP: imitate different pre-recorded human animations)

Ingenuity (PPO)

Quadcopter (PPO)

ShadowHand (PPO)

[Toni-SM]

NVIDIA Omniverse Isaac Gym

---

Benchmark

Setup

Package	Version
Omniverse Isaac Sim	2022.1.1
OmniIsaacGymEnvs	commit d0eaf2e7f1e1e901d62e780392ca77843c08eb2c

Results

Environment	PPO
Allegro Hand 1	2380.96
Ant	6131.46 +/- 579.04
Anymal	54.21 +/- 2.46
Anymal Terrain 2	11.87 +/- 0.13
Ball Balance	338.97 +/- 26.59
Cartpole 2	496.53 +/- 1.9
Crazyflie	1106.75 +/- 63.75
Franka Cabinet	2383.55 +/- 449.01
Humanoid	6550.07 +/- 619.16
Ingenuity	5251.26 +/- 92.89
Quadcopter	1200.52 +/- 9.03
Shadow Hand 1	11175.08

¹ one run
² variable behavior to the random number generator seed

Comparison (single run)

• rl_games-v1.5.2
  • AllegroHand: PPO (4198)
  • Ant: PPO (6694)
  • Anymal: PPO (56)
  • AnymalTerrain: PPO (11)
  • BallBalance: PPO (367)
  • Cartpole: PPO (350)
  • Crazyflie: PPO (1069)
  • FrankaCabinet: PPO (3272)
  • Humanoid: PPO (6855)
  • Ingenuity: PPO (5285)
  • Quadcopter: PPO (1193)
  • ShadowHand: PPO (9904)

Visualizations

The following charts show the episode's mean length in timesteps (left) and the mean total reward (right)

Allegro Hand (PPO)

Ant (PPO)

Anymal (PPO)

Anymal Terrain (PPO)

Ball Balance (PPO)

Cartpole (PPO)

Crazyflie (PPO)

Franka Cabinet (PPO)

Humanoid (PPO)

Ingenuity (PPO)

Quadcopter (PPO)

Shadow Hand (PPO)

[Toni-SM]

Gym / Gymnasium

---

Benchmark

Setup

Package	Version
rl-zoo3	1.7.0
pybullet	3.2.5

Results

Environment	DDPG	TD3	SAC
AntBulletEnv-v0	2666.89 +/- 390.55	3315.37 +/- 164.7	2908.14 +/- 349.64
HalfCheetahBulletEnv-v0	2586.13 +/- 180.78	2650.21 +/- 222.45	2521.31 +/- 100.81
HopperBulletEnv-v0	109.05 +/- 38.67	2328.89 +/- 248.81	in-progress
Walker2DBulletEnv-v0	122.65 +/- 41.93	1665.87 +/- 268.12	in-progress

Comparison (single run)

• stable-baselines3-v1.7.0
  • AntBulletEnv-v0: DDPG (2416), TD3 (3301), SAC (3243)
  • HalfCheetahBulletEnv-v0: DDPG (2034), TD3 (2821), SAC (3449)
  • HopperBulletEnv-v0: DDPG (245), TD3 (2105), SAC (1722)
  • Walker2DBulletEnv-v0: DDPG (135), TD3 (1744), SAC (2103)

Visualizations

The following charts show the episode's mean length in timesteps (left) and the mean total reward (right)

AntBulletEnv-v0 (DDPG: gray, TD3: blue, SAC: pink)

HalfCheetahBulletEnv-v0 (DDPG: gray, TD3: blue, SAC: pink)

HopperBulletEnv-v0 (DDPG: gray, TD3: blue, SAC: pink)

Walker2DBulletEnv-v0 (DDPG: gray, TD3: blue, SAC: pink)

---

Recurrent Neural Network showcase

Setup

Package	Version
gym	0.21.0

Results

Environment	DDPG	TD3	SAC	PPO	TRPO1
Pendulum	-86.13 +/- 86.79	-224.74 +/- 83.34	-122.64 +/- 67.52	-154.67 +/- 48.34	-254.86 +/- 85.52
PendulumNoVel	-1153.45 +/- 236.88	-1094.73 +/- 224.47	-999.44 +/- 389.66	-1162.71 +/- 136.45	-1176.2 +/- 124.0
PendulumNoVel (RNN)	-1275.12 +/- 353.08	-1497.62 +/- 167.72	-1270.12 +/- 226.81	-1351.63 +/- 102.25	-620.77 +/- 42.04
PendulumNoVel (GRU)	-227.39 +/- 114.9	-154.98 +/- 92.79	-219.43 +/- 77.77	-274.28 +/- 60.07	-723.86 +/- 91.34
PendulumNoVel (LSTM)	-235.3 +/- 145.1	-201.74 +/- 62.01	-167.88 +/- 58.08	-167.88 +/- 58.08	-751.76 +/- 103.22

Visualizations

The following charts show the episode's mean total reward

Pendulum, PendulumNoVel

PendulumNoVel (RNN), PendulumNoVel (GRU), PendulumNoVel (LSTM)

[Toni-SM]

NVIDIA Isaac Orbit

---

Benchmark

Setup

Package	Version
Omniverse Isaac Sim	2022.2.0
Isaac Orbit	commit a66d384cccbe696f83915febbff3e8a71fffcef1

Results

Environment	PPO
Isaac-Ant-v0	11336.27 +/- 182.79
Isaac-Cartpole-v0 1	497.72 +/- 0.79
Isaac-Humanoid-v0 1	5401.19 +/- 387.51
Isaac-Lift-Franka-v0 env	-
Isaac-Reach-Franka-v0	9.1 +/- 0.05
Isaac-Velocity-Anymal-C-v0 env	-

¹ variable behavior to the random number generator seed
^env waiting for the environment to be adjusted

Comparison (single run)

• rl_games-v1.5.2
  • Isaac-Ant-v0: PPO (11500)
  • Isaac-Cartpole-v0: PPO (493.1)
  • Isaac-Humanoid-v0 (schedule_type: standard): PPO (6489)
  • Isaac-Reach-Franka-v0: PPO (8.77)

Visualizations

The following charts show the episode's mean length in timesteps (left) and the mean total reward (right)

Isaac-Ant-v0 (PPO)

Isaac-Cartpole-v0 (PPO)

Isaac-Humanoid-v0 (PPO)

Isaac-Reach-Franka-v0 (PPO)