Background
We already have a grouped set of physics-step benchmark entrypoints under benchmark/:
benchmark/benchmark_physics_step_mj_step.py
benchmark/benchmark_physics_step_genesis.py
benchmark/benchmark_physics_step_isaacgym.py
benchmark/benchmark_physics_step_mujoco_warp.py
But we still lack a clear, repo-level benchmark target / support claim for comparing physics_step throughput across backends, especially for genesis, isaacgym, and mujoco_warp.
Goal
Add benchmark coverage and a stable reporting path for cross-backend physics_step throughput, including at least:
genesis
isaacgym
mujoco_warp
- baseline MuJoCo path (
benchmark_physics_step_mj_step.py)
Expected Scope
- Standardize the measurement target as physics-step throughput / latency, not full env.step throughput.
- Ensure the backend-specific benchmark scripts expose comparable output fields and compatible task / batch-size sweeps.
- Save results under the shared
benchmark/outputs/physics_step/ subtree so results are easy to compare and postprocess.
- Make it easy to compare:
- total throughput
- per-env throughput
- latency per benchmark call
- Keep task naming aligned with current owner task ids (
go1_joystick, go2_joystick, g1_joystick) and avoid stale benchmark-only naming drift.
Validation
- Run the relevant
benchmark_physics_step_* scripts for at least one overlapping task / batch-size sweep.
- Confirm JSON outputs are produced under
benchmark/outputs/physics_step/....
- Confirm the reported metrics are comparable across backends.
Notes
This issue is about benchmark coverage and evidence for support / performance claims, not about changing the env contract or training entrypoints.
Background
We already have a grouped set of physics-step benchmark entrypoints under
benchmark/:benchmark/benchmark_physics_step_mj_step.pybenchmark/benchmark_physics_step_genesis.pybenchmark/benchmark_physics_step_isaacgym.pybenchmark/benchmark_physics_step_mujoco_warp.pyBut we still lack a clear, repo-level benchmark target / support claim for comparing physics_step throughput across backends, especially for
genesis,isaacgym, andmujoco_warp.Goal
Add benchmark coverage and a stable reporting path for cross-backend
physics_stepthroughput, including at least:genesisisaacgymmujoco_warpbenchmark_physics_step_mj_step.py)Expected Scope
benchmark/outputs/physics_step/subtree so results are easy to compare and postprocess.go1_joystick,go2_joystick,g1_joystick) and avoid stale benchmark-only naming drift.Validation
benchmark_physics_step_*scripts for at least one overlapping task / batch-size sweep.benchmark/outputs/physics_step/....Notes
This issue is about benchmark coverage and evidence for support / performance claims, not about changing the env contract or training entrypoints.