Skip to content

feat(microduck): port BAM xl330-m6 voltage actuator model (issue #1474) - #1475

Merged
TATP-233 merged 2 commits into
dev/issue-1042-manager-based-apifrom
feat/issue-1474-microduck-bam-actuator
Sep 3, 2026
Merged

feat(microduck): port BAM xl330-m6 voltage actuator model (issue #1474)#1475
TATP-233 merged 2 commits into
dev/issue-1042-manager-based-apifrom
feat/issue-1474-microduck-bam-actuator

Conversation

@TATP-233

@TATP-233 TATP-233 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator

Fixes #1474

背景

#1452 对齐对比的根因分析(#1464 评论区)确认:microduck 与上游 microduck_rl 的剩余训练差距主因是执行器模型不同(原因 1)——UniLab 用 XML position PD(kp=50/kv=0.5),上游用 BAM xl330-m6 电压舵机。本 PR 把 BAM 模型移植到 UniLab 侧,使执行器动力学不再是对齐差距的来源。

实现

BamVoltageActionsrc/unilab/tasks/locomotion/microduck/bam_action.py):task-owned action term,requires_substep_state_feedback=True,经 SimBackend 已声明的 set_pre_step_control 契约在每个物理 substep 重算力矩(go2w 先例),零 SimBackend / unisim-core 契约变更。逐 substep 管线与上游 bam.mjlab.BamActuator 逐项对应:

  • LIFO 指令延迟环形缓冲(lag ~ U{3..6} substep = 15–30 ms,每 substep 重采,reset 回填)
  • 每 env 电池模型:vin ~ U(6.5, 8.2) V、vin_drop_gain ~ U(0, 0.2) V/Nm(startup 采样、跨 reset 保持),vin_eff 下限 6.0 V
  • 固件 P 律(kp_fw=200,XL330 error_gain)+ 1.75 A 电流限窗 + PWM 限幅
  • back-EMF 力矩 τ=(kt·V−kt²·dq)/R,forcerange ±1.0676 Nm 由 XML <motor> 截断
  • m6 摩擦预算全项 × friction_scale ~ U(0.9,1.1)(每 reset 赋值式重采)
  • encoder_bias 在延迟缓冲之前扣除,与 PD 配方的 JointPositionAction 同序

新 task MicroduckVelocityBamFlat(仅 mujoco owner;mjwarp host_numpy profile 显式拒绝 pre-step control,fail-closed)复用 PD 配方的 reward/obs/curriculum/commands/terminations 逐项不变(owner 契约测试断言相等),执行器模型是唯一变量。

与上游的近似边界(模块 docstring 同步记录)

  1. 上游把摩擦预算写入 dof_frictionloss/dof_damping 由求解器做静摩擦 clip;UniLab SimBackend 契约无逐 substep 写这两个字段的通道,故预算折算进输出力矩,stiction clip 在力矩域复刻。
  2. 上游用 qfrc_bias/qfrc_constraint/qfrc_friction 估计外部力矩;契约拿不到 qfrc_*,改用有限差分 I_eff·a_fd − τ_applied_prev(I_eff=名义 dof_armature)。稳态无差,快速暂态略低估惯性反作用。

探针对拍(64 envs,seed 42,UniLab BAM/mujoco vs 上游 BAM/mjwarp 既有探针参照值)

指标 UniLab BAM 上游参照 偏差
零 action 姿态保持:倾倒 env 数 64/64(356 次) 64/64(383 次) 一致
平均首次倾倒时刻 64.1 步(~1.28 s) 64 步(~1.3 s) <1%
pg_xy 尾段均值 0.270 0.276 -2%
膝关节 +0.2 rad 阶跃 t90 403 ms ~370 ms +9%(含 20 ms 步长量化)
阶跃稳态误差 -0.076 rad(38% 下垂) -0.068 rad(34%) 同方向同量级
零 action 稳态关节误差(总体/膝/踝) 0.036 / 0.118 / 0.056 rad 0.049 / 0.148 / 0.084 rad 系统性偏硬 ~25%(力矩域摩擦近似所致)

含延迟、电压 sag、限幅的动态指标(A/B)几乎复刻;静态误差(C)偏硬约 25%,来源于上述力矩域摩擦近似——已在代码与本文档记录。

Validation

  • make test-all 于最终 head 8371566 通过(2583 passed;benchmark module/script-mode 仅 platform-optional mlx 跳过)。
  • 新增 18 个单测(电压律/电流限窗/PWM 限幅/back-EMF 数值用例、延迟缓冲语义、摩擦预算、stiction clip、reset 语义、fail-closed 校验、owner materialize 与 PD owner recipe 逐项相等断言、真实 mujoco 冒烟)。
  • ruff / mypy / pyright 零告警;support matrix 已重新生成。

训练对比

BAM task 修复验证训练(mujoco CPU,4096 envs × 2000 iter,seed 42)进行中,结果以本 PR 评论跟进(上游 seed42 run 复用 #1452 既有数据)。

NumPy port of the upstream bam.mjlab.BamActuator (xl330-m6 voltage servo)
as a task-owned action term on the Manager-Based runtime, recomputing
motor torque every physics substep through the SimBackend
set_pre_step_control contract (go2w precedent): LIFO command-delay ring
buffer (lag 3-6 substeps), per-env battery model (vin 6.5-8.2 V startup
DR with load-dependent drop), firmware P law with current-limit duty
window and PWM clip, back-EMF torque equation, and the m6 friction
budget (Coulomb + Stribeck + directional load + quadratic) under per-env
friction_scale reset DR.

Approximation boundaries vs upstream (documented in the module
docstring): the friction budget is folded into the output torque with a
torque-domain stiction clip because the SimBackend contract has no
per-substep dof_frictionloss/dof_damping write channel, and the external
torque is a finite-difference estimate since qfrc_* is not exposed.

New task MicroduckVelocityBamFlat (mujoco owner only; the mjwarp
host_numpy profile rejects pre-step control) reuses the PD recipe's
reward/obs/curriculum verbatim so the actuator model is the only delta.

Probe cross-check vs upstream (64 envs, seed 42): zero-action falls
356/64 envs with mean first fall at 64.1 steps (upstream 383, 64),
knee-step t90 403 ms (upstream ~370 ms), steady-state error overall
0.036 rad / knee 0.118 rad (upstream 0.049 / 0.148; stiffer by ~25% from
the torque-domain friction approximation).
@TATP-233

TATP-233 commented Sep 3, 2026

Copy link
Copy Markdown
Collaborator Author

BAM 训练对齐结果(seed42,4096 envs × 2000 iter,mjwarp 两侧)

修复内容:将 microduck 执行器从 PD 配方改为上游原版 BAM(BLDC + 母线电流环,substep 电压积分),对应 #1464 根因分析的原因 1(执行器模型不同)。

总览(终段窗口均值,UniLab vs 上游)

指标 PD 配方(修复前) BAM(本 PR) 上游
mean_reward 83.8(-26.4%) 112.098(-1.5%) 113.833
episode_length 956.5 944.6
收敛 iter 231 128

Reward term 对比(BAM vs 上游)

term UniLab BAM 上游 差异 PD 配方曾差
tracking_lin_vel 1.253 1.267 -1.1%
tracking_ang_vel 0.628 0.679 -7.4% -96.6%
leg_pose 0.651 0.608 +6.9% -89.7%
body_ang_vel -0.031 -0.031 0 11×
upright 1.736 1.713 +1.3%
dof_pos_limits -0.000 -0.001
foot_swing_height -0.014 -0.003 4.3×
foot_clearance -0.006 -0.004 -40.4%
air_time 0.978 1.024 -4.6%
head_pose_tracking 1.695 1.705 -0.6%

终止占比:tilt 10.1% vs 11.3%,无 NaN。

结论

  1. 执行器对齐后,所有结构性差距收敛到 seed 噪声量级:mean_reward 从 -26.4% 收窄到 -1.5%,tracking_ang_vel / leg_pose / body_ang_vel 等此前差 1~2 个数量级的 term 全部归位。原因 1 被证实并消除。
  2. 残差主项是 foot_swing_height(4.3×)与 foot_clearance(-40%),与探针协议 C 测得的 BAM 稳态偏硬 ~25% 同源——当前用 forcerange ±1.0676 的力矩域摩擦近似求解器级摩擦,非完全等价。
  3. 继续压缩残差需要求解器层的摩擦通道(unisim-core 契约扩展)以及 mjwarp 对 BAM 路径的支持,属于后续 roadmap 候选,不在本 PR 范围。

产物

  • 训练日志:logs/rsl_rl_ppo/MicroduckVelocityBamFlat/2026-09-04_00-03-14_mujoco(seed42,exit 0,48 分钟)
  • 对比报告:logs/alignment_1474/report_bam_seed42.md / .jsonscripts/microduck_alignment_compare.py 生成;上游复用 microduck_rl/logs/rsl_rl/velocity/2026-09-03_19-48-28_seed42
  • 两侧各 n=1 seed,数值差异含 seed 噪声,解读时以数量级/结构性为准。

Validation

  • make test-all @ head 83715664:2583 passed
  • 新增 18 个 BAM 单测全绿

@TATP-233
TATP-233 merged commit 3ba89e3 into dev/issue-1042-manager-based-api Sep 3, 2026
@TATP-233
TATP-233 deleted the feat/issue-1474-microduck-bam-actuator branch September 3, 2026 17:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant