Skip to content

LocoMuJoCo v1.2.0

Latest

Choose a tag to compare

@robfiras robfiras released this 18 Nov 20:11
· 1 commit to master since this release

LocoMuJoCo now includes SuperMimic and supports real-robot deployment on the AgiBot X2 Ultra and the Unitree G1 through LocoMuJoCo Real.

🤖 Highlight: from LocoMuJoCo to the real robot

The AgiBot X2 Ultra running a tracking policy, on the robot and in simulation

LocoMuJoCo Real is the companion repository for taking policies trained in LocoMuJoCo to a real robot. It covers the Unitree G1 (G1Real) and the AgiBot X2 Ultra (X2UltraReal); both pipelines have been run on the robot.

  • Deployable training setups. For each robot an environment with position servos, the observations the robot can measure, a tracking reward and sim-to-real randomization, with ready-to-run configs for motion tracking with PPO and with SuperMimic:

    python training/<robot>/train.py             # PPO
    python training/<robot>/train_supermimic.py  # SuperMimic
  • One path from checkpoint to robot, the same for both algorithms. Each stage removes one source of error:

    1. export the policy to a bundle that runs with only numpy and mujoco (deployment/export_bundle.py),
    2. validate the bundle against the training environment, term by term (deployment/validate_bundle.py),
    3. roll it out in plain MuJoCo (deployment/rollout_mujoco.py),
    4. sim-to-sim over the robot's middleware, Unitree SDK 2 (DDS) for the G1 and AimDK (ROS 2) for the X2 (deployment/<robot>/sim2sim_bridge.py), driven by the controller that later runs on the robot,
    5. the robot, hanging first (deployment/g1/deploy_g1.py, deployment/x2/deploy_x2.py).
  • Motion data. LocoMuJoCo's default clips and LAFAN1 are retargeted to the robot on first use. LocoMuJoCo Real also documents how to load your own trajectories and how to convert and train on BONES-SEED.

The real robot deployment tutorial explains the stages and how to bring up another robot.

Warning

Running a learned policy on a real robot is hazardous. Hang the robot from a crane or gantry, keep the emergency stop in hand, and run every policy in sim-to-sim first.

🚀 SuperMimic

  • SuperMimicJax (from loco_mujoco.algorithms import SuperMimicJax): whole-body tracking with one differentiable world model, following SuperMimic. A world model is fitted on replayed transitions and the policy is improved by backpropagating through imagined rollouts, with a TD(λ) critic.
  • Imagination reuses the environment's own JAX observation, reward and termination functions, so the algorithm is agnostic to the robot, the observation and the reward.
  • The environment steps run on MJX or on MjWarp (use_mjwarp=True); under domain randomization the randomizer state is given to the dynamics model and the critic as context.
  • Training example on the Unitree G1 with eval.py and a per-clip evaluator eval_clips.py, and the API documentation.
  • New building blocks that came with it: goals TrajFutureFrames, RootOriRelTraj and TrackingError, and the BeyondMimic-style SiteTrackingReward, which tracks the mimic sites and so needs no robot-specific body list.
  • Training runs at about 90 % of the speed of XMimic on the same hardware (Unitree G1, 4096 envs on MjWarp, 40 LAFAN1 clips, one RTX 3090).

Acknowledgements. Our implementation was written with BlinkMimic and XMimic as references. Thanks to their authors for making them available.

⚡ Simulation backends

  • MjWarp is no longer flagged experimental; the warning and the 3 s pause on creation are gone.
  • Fixed the MjWarp reset: an ended episode now restores every per-world field (before, only qpos and qvel were reset, which broke tasks without a reference trajectory).
  • Under MjWarp the per-world index is available as carry.env_id, so vmapped rewards can attribute the shared contact buffer (contact__worldid) to their own world.
  • naconmax is forwarded to MjWarp's data; models whose collision pairs the MJX JAX backend does not implement (e.g. the full-collision G1) can be created with MjWarp.
  • MuJoCo 4 is supported (MjWarp data creation, retargeting welds, viewer camera).
  • model_contact_conf=dict(self_collisions=False, condim=...) changes the robot's contact setup at construction.
  • examples/speed_test.py is now a throughput benchmark across MuJoCo, MJX and MjWarp with reduced and full collision models.

⛰️ Terrain

  • HeightmapTerrain: a static terrain loaded from a height map .npz, aligned with the world frame, for datasets whose clips are placed on the map. Collision through heightfield tiles, boxes or a per-env box pool.
  • FlatTerrain is the new name of the default flat floor. StaticTerrain and DynamicTerrain remain the base classes: FlatTerrain and HeightmapTerrain are static, RoughTerrain is dynamic.
  • HeightScan observation: terrain heights on a grid around the root, in the root's heading frame.
  • Terrain overview and API documentation.
  • All terrains share one height query (get_height_at) and a documented lifecycle, and decide where episodes start. recenter_root_xy controls whether episodes start at the origin or at the xy of the clip.
  • MjWarp shares heightfield data across worlds, so it now refuses terrains that write a heightfield per env (RoughTerrain) with an error instead of simulating the wrong terrain. Use HeightmapTerrain there.

🎞️ Trajectories and datasets

  • The trajectory is an argument, not environment state. ImitationFactory.make returns (env, traj), and mjx_reset, mjx_step, reset and step accept traj. Trajectories are padded to fixed shapes, so a jitted training function can be called with a different set of clips without recompiling (see Breaking changes).
  • Trajectory handler registry. TrajectoryHandler subclasses register like the other components and are selected with th_params=dict(name=...): RandomStartTrajectoryHandler (default), RandomTrajFixedStepTrajectoryHandler and FixedStartTrajectoryHandler. The old random_start / fixed_start_conf keys keep working with a DeprecationWarning.
  • On-the-fly retargeting for new robots. If a default or LAFAN1 clip is not hosted for an environment, it is retargeted from a source environment on first use and cached. loco-mujoco-retarget-for-new-env --env <Env> does this ahead of time for all clips.
  • Robot confs can set joint_limit_solref / joint_limit_solimp for retargeting.
  • The smpl extra installs smplx from PyPI instead of a fork.
  • GoalTrajMimic and MimicReward accept sites_for_mimic and main_site_name.

📦 Installation and packaging

  • LocoMuJoCo is on PyPI: pip install loco-mujoco. The examples and tutorials are in the repository.
  • Robot models are downloaded on first use (about 200 MB, cached in ~/.loco-mujoco-caches/robot_models) instead of being a git-URL dependency. loco-mujoco-download-models fetches them ahead of time, and LOCOMUJOCO_ROBOT_MODELS_PATH points to a local copy for machines without network access.
  • JAX is pinned to jax<=0.10.1,!=0.10.0: JAX 0.10.0 and 0.10.2 fail to compile SuperMimic's training function on the GPU.
  • Only the loco_mujoco package is installed; docs and scripts no longer end up as top-level packages.
  • New extras: viser, usd, blender, test. requests, httpx, rich and filelock are declared.
  • New CLI: loco-mujoco-show-variable (--name or --all).

🖥️ Rendering

  • Browser rendering with viser: env.render_viser(), env.mjx_render_viser(state) and play_trajectory(viser=True) serve the scene to a browser, which helps over SSH and on machines without a display (pip install -e ".[viser]").
  • USD export: env.render_usd(), env.mjx_render_usd(state) and play_trajectory(usd=True) write an animated USD stage (pip install -e ".[usd]"); scripts/blender renders it with Blender.
  • Rendering tutorial, tutorials 12_viser_rendering.py and 13_viser_rendering_mjx.py.
  • The observation summary is printed in the terminal when an environment is created (print_obs_summary=False turns it off). create_observation_summary writes a local HTML file instead of uploading it.

🧠 Algorithms

  • init_agent_state separates network initialization from training, and one build_train_fn covers fresh runs and resuming, so training can run in chunks (examples/training_examples/jax_rl_mimic/experiment_resume.py).
  • Reward normalization statistics moved from an environment wrapper into the agent state, so they are saved with the agent.
  • Multi-GPU PPO: with experiment.n_devices the training function runs under jax.pmap and averages the gradients across devices. training/ppo.py in LocoMuJoCo Real is the reference driver.
  • Observation history can be limited to one group (NStepWrapper(group=...), obs_history_group), so the critic is not stacked; get_obs_ind_by_group accepts a list of groups.
  • New in loco_mujoco.algorithms.experimental: S2PGPPOJax, BPTTPPOJax, HistoryPPOJax, SACJax, TD3Jax and VanillaDaggerJax (DAgger distillation), each with a training example.

📚 Documentation and tests

  • New tutorials: Adding a New Robot, Real Robot Deployment and Rendering in the Browser and Exporting to USD.
  • The API documentation now covers SuperMimic, and the terrain page starts with an overview of the terrains.
  • A much larger test suite. CI also runs on pull requests into master and dev, installs the optional extras and reports coverage.
  • Fixes found on the way: the Force observation, mat2angle_xy / angle2mat_xy, muscle peak-force scaling of the skeleton models, randomize_link_mass=False, forward kinematics after the initial-state handler, a viewer crash when the window is closed, and the torch_device argument of the SMPL motion transfer.

⚠️ Breaking changes and migration

v1.1.0 v1.2.0
env = ImitationFactory.make(...), env = RLFactory.make(...) env, traj = ImitationFactory.make(...); RLFactory.make returns (env, None)
from loco_mujoco.trajectory import ... from loco_mujoco.core.trajectory import ...
env.load_trajectory(traj) traj = env.process_trajectory(traj)
TrajectoryHandler(model, traj=..., control_dt=...), env.th.traj the handler holds only the TrajectoryInfo; use the trajectory returned by the factory or by process_trajectory, and TrajectoryHandler.process(traj, model, dt) to fit a trajectory to a model
train_fn = PPOJax.build_train_fn(env, agent_conf); train_fn(rng) agent_state = PPOJax.init_agent_state(env, agent_conf, rng); train_fn(rng, agent_state, traj) (same for GAIL and AMP). With several seeds: jax.vmap(train_fn, in_axes=(0, 0, None))
PPOJax.play_policy(env, agent_conf, agent_state, ...) add traj=traj for imitation tasks
NormalizeVecReward, NormalizeVecRewEnvState wrappers removed; the algorithms normalize the reward themselves
th_params=dict(random_start=False, fixed_start_conf=(0, 0)) still works, deprecated; use th_params=dict(name="FixedStartTrajectoryHandler", start_conf=(0, 0))
LOCOMUJOCO_CUSTOM_SMPL_CONF_PATH LOCOMUJOCO_CUSTOM_ROBOT_CONF_PATH (set it again with loco-mujoco-add-variable)
DefaultDatasetConf(dataset_type="pretrained") removed, only "mocap" exists; anything else raises a ValueError
create_observation_summary() returns a URL returns the path of a local HTML file
  • env.mjx_reset(key) and env.mjx_step(state, action) still work without traj and then use the trajectory the environment was created with. Pass traj (jax.vmap(env.mjx_reset, in_axes=(0, None))) to be able to swap it.
  • Dependencies: jax<=0.10.1,!=0.10.0 (GPU: pip install "jax[cuda12]<=0.10.1,!=0.10.0"). The loco-mujoco-models git dependency is gone; the models are fetched on first use, so offline machines need loco-mujoco-download-models beforehand or LOCOMUJOCO_ROBOT_MODELS_PATH.
  • Python: unchanged, 3.10 or newer.
  • MjWarp with RoughTerrain now raises an error at construction.
  • The default terrain_type is "FlatTerrain" instead of "StaticTerrain". Both give the same flat floor, and terrain_type="StaticTerrain" keeps working.
  • Removed: the console scripts loco-mujoco-download, -download-real and -download-perfect (they pointed at functions that did not exist), and the unused CollisionExtender, optimize_for_collisions and MANO_Parser.
  • Environments print their observation summary on creation; pass print_obs_summary=False to keep logs quiet.

🙏 Contributors

Thanks to @mdrolet01 for the trajectory handler registry, the handler's backend handling and the quadruped init-state fix (#92, #95, #96).

Full changelog: v1.1.0...v1.2.0