feat(microduck): add bilateral mirror loss and keep turn-in-place commands

[alexhegit]

Summary

• Add RSL-RL mirror loss (use_data_augmentation=false) for the MicroDuck 61D/14D policy contract, on both the PD and BAM owners. Without it, MuJoCo position-PD PPO converges to seed-dependent one-sided turning.
• Keep the 15% turn-in-place command bucket from being zeroed by the base standing sampler.
• Do not change the BAM actuator model or the microduck 对齐 child 3/5: reward / MDP 栈对齐到上游当前 HEAD #1455 reward stack. Multi-GPU env counts stay out of owner YAML.

Builds on #1475 (BAM) and #1455 (reward stack).

Scope

In

• microduck_velocity_symmetry plus involution / lateral-and-yaw sign-flip tests
• symmetry_cfg on microduck_velocity_flat/mujoco.yaml and microduck_velocity_bam_flat/mujoco.yaml
• MicroduckVelocityCommand._resample_command: is_standing_env[turn_ids]=False and refresh vel_command_w
• Contract tests: PD owner enables mirror loss; BAM and PD share the same symmetry function

Out

• Defaults for algo.num_envs, max_iterations, and training.devices (BAM remains 2048×500 smoke; PD remains 4096×6000)
• BAM voltage servo, XML, and substep torque (feat(microduck): port BAM xl330-m6 voltage actuator model (issue #1474) #1475)
• Local ROCm pyproject.toml / uv.lock
• Checkpoints and mp4 (logs/)

Local training notes (not in-tree)

On a 4×MI210 host with a MuJoCo CPU pool, algo.num_envs is per rank:

uv run train --algo ppo --task microduck_velocity_bam_flat --sim mujoco \
  algo.num_envs=2048 algo.max_iterations=12000 \
  'training.devices=[0,1,2,3]'

Global 8192 improved throughput versus 1024/rank and improved PD omnidirectional coverage on this machine. That is host CPU-thread evidence, not a cross-machine default.

BAM 12k (2026-09-04_12-41-58): no NaN; error_vel_yaw 2.13 (PD 12k on the same host was ~9.7); fixed-command stand yaw ≈0, turn-left +0.60 / turn-right −0.72 vs ±0.7. Forward/lateral tracking is still weak and is out of this PR.

Tightening tracking_ang_vel sigma 0.5→0.25 on a PD resume lowered non-yaw rewards as well; it is not adopted.

Validation

• make check (ruff / mypy / pyright) passed with Node 24 (PATH via nvm; system Node 12 cannot run pyright).
• uv run pytest tests/envs/locomotion/microduck tests/envs/locomotion/test_sensor_reward_terms.py -m "not slow": 63 passed.
• uv run python scripts/benchmark/smoke_test.py: passed (mlx scripts skipped as platform-optional).
• Full make test-all test-cov: 2541 passed. Remaining failures/errors on this host were missing HF meshes (go2/a2/allegro recovered after unilab-pull-assets; t800 still missing LINK_HIP_PITCH_R.obj) and off-policy logger tests asserting untruncated labels under a narrow captured terminal. None of those paths are in this diff.

Made with Cursor

[TATP-233]

全部迁移至
https://github.com/unilabsim/microduck_rl_unilab