TL;DR – I have a Unitree G1 hierarchical controller trained in Isaac Lab (PhysX):
a high-level switcher → 64-D "target-frame" AMP actors (run/jump) → a FROZEN BeyondMimic/ProtoMotions tracker (157-D obs → 29 joint targets, with BeyondMimic PD scaling). Re-implementing this in MuJoCo, the tracker alone matches a reference motion fine, but the combined policy falls in ~1 s. Matched the model, gains, armature, default pose, timing, obs/action conventions exactly and ruled out all the usual suspects. Looking for anyone who has actually transferred an Isaac/PhysX-trained tracker + RL actor stack to MuJoCo (or knows the missing piece).
--- STACK ---
- PhysX 2000 Hz (dt=0.005), decimation 4 → 50 Hz control.
- Actuation: implicit position PD (Isaac "built-in PD"), stiffness/damping = BeyondMimic G1 values, effort limits per joint, armature per joint.
- Low-level (explicable) tracker:
proprio(64) = dof_pos(29) + dof_vel(29) + root_local_ang_vel(3) + proj_gravity(3)
target(64) = rel_anchor_rot6d(6) + dof_vel(29) + proj_gravity(3)
prev(29) = previous processed actions (absolute joint targets)
obs = concatenate(proprio, target, prev) → 157; action → q_target = default_dof_pos + (effort/stiffness)*raw
- Mid-level: AMP run actor, obs 134 = proprio(64) + base_lin_vel(3) + vel_cmd(3) + prev_target_frame(64); outputs the 64-D target frame directly.
- High level: 6-D switcher → gets run/jump.
- Model: g1_holo_output.xml + same MJCF used for tracking; per-joint armature, frictionloss=0.1, sphere feet).
--- MuJoCo REPRO (what we matched) ---
- Loaded the same MJCF; set opt.timestep=0.005; 4 substeps @ 50 Hz.
- Same default pose (hip -0.312, knee 0.669, ankle -0.363, elbow 0.6, shoulders +/-0.2), same stiffness/damping/effort, same armature.
- Same obs conventions: local ang vel = pelvis-frame qvel[3:6]; proj gravity in torso frame; normalization baked into the exported .pt.
- Same action scaling (offset + effort/stiffness * raw) and same prev_action processed = absolute targets.
- Cross-checked with the tracker's embedded normalization constants.
--- EVIDENCE ---
- Tracker alone: feeds target frames built from a real reference locomotion clip → robot stays upright for the full episode in MuJoCo (z ≈ 0.78, no fall). So the tracker checkpoint, model, gains and plumbing are OK in MuJoCo.
- Composite (run actor + tracker): falls in ~0.9–2 s for every velocity command (including cmd=0), from every initial joint pose (zeros, default), with prev-action zero or warm-started.
- Diagnostics: the exported run actor's obs_mean_action block does not look like a valid target frame (non-unit rot6d, and even under a kinematic replay of a reference motion the actor saturates to a constant, out-of-distribution frame. Target-frame block-order permutations and rot6d orthonormalization don't help.
--- WHAT WE TRIED (no fix) ---
- sim dt 0.001 vs 0.005; explicit torque PD vs implicit position PD.
- friction loss on/off; contact solref/solimp; friction; foot capsules vs spheres; self-collision on/off.
- EMA on actions (1.0/0.8/0.5/0.3/0.20), PD-target clamp, exact-effort clipping.
- Target-frame layout permutations, identity rot6d, command=0, init-pose/warm-start sweeps.
- A proven Open MuJoCo G1 deploy harness (RoboJudo) as the environment: its own ProtoMotions tracker stands and tracks, ours does too WITH REFERENCE MOTIONS, but the composite run actor still falls.
--- CONTEXT / PRIOR ART ---
--- QUESTION ---
- Has anyone successfully deployed an Isaac/PhysX-trained BeyondMimic/ProtoMotions tracker with an RL actor on top to MuJoCo without retraining? If so, what contact/actuator/system-ID details were decisive?
- Is the composite's failure expected (closed-loop fragility of the aggressive actor) or is there a known convention mismatch (e.g., target-frame parameterization, reduced-coords target built relative vs absolute, or per-joint target index order) that we're missing?
- Does anyone know a public, smaller run/jump motion pack (or a working MuJoCo sim2sim script) for this exact tracker so we can cross-check the run actor's expected target distribution?
Reproduction artifacts at: https://github.com/karvachiik-lgtm/g1-isaac-mujoco-sim2sim-repro
TL;DR – I have a Unitree G1 hierarchical controller trained in Isaac Lab (PhysX):
a high-level switcher → 64-D "target-frame" AMP actors (run/jump) → a FROZEN BeyondMimic/ProtoMotions tracker (157-D obs → 29 joint targets, with BeyondMimic PD scaling). Re-implementing this in MuJoCo, the tracker alone matches a reference motion fine, but the combined policy falls in ~1 s. Matched the model, gains, armature, default pose, timing, obs/action conventions exactly and ruled out all the usual suspects. Looking for anyone who has actually transferred an Isaac/PhysX-trained tracker + RL actor stack to MuJoCo (or knows the missing piece).
--- STACK ---
proprio(64) = dof_pos(29) + dof_vel(29) + root_local_ang_vel(3) + proj_gravity(3)
target(64) = rel_anchor_rot6d(6) + dof_vel(29) + proj_gravity(3)
prev(29) = previous processed actions (absolute joint targets)
obs = concatenate(proprio, target, prev) → 157; action → q_target = default_dof_pos + (effort/stiffness)*raw
--- MuJoCo REPRO (what we matched) ---
--- EVIDENCE ---
--- WHAT WE TRIED (no fix) ---
--- CONTEXT / PRIOR ART ---
--- QUESTION ---
Reproduction artifacts at: https://github.com/karvachiik-lgtm/g1-isaac-mujoco-sim2sim-repro