feat(microduck): 训练基建对齐上游(env RNG 播种 / init_at_random_ep_len / 规模与 seed) - #1462
Merged
TATP-233 merged 2 commits intoSep 3, 2026
Conversation
- thread env-level RNG seed through the three microduck mjwarp owners (env.seed=42): the Hydra -> BackendAdapter -> registry chain already reaches ManagerBasedRlEnvCfg.seed, so command/noise/delay/DR sampling is now reproducible across runs; unset owners keep the None default - fix init_at_random_ep_len: RslRlVecEnvWrapper.episode_length_buf becomes a property whose setter propagates into the env's real episode counters via the new ManagerBasedRlEnv.set_episode_length_buf cold-path entry (keeps episode_length_buf and state.info["steps"] in sync); upstream mjlab's wrapper setter writes the env buffer directly, so staggering is effective upstream and now matches here - scale alignment: algo.num_envs 2048 -> 4096 and algo.seed 1 -> 42 in all three mjwarp owners; max_iterations 500 -> 2000 (2000 x 24 = 48000 env steps, exactly covering every curriculum terminal stage; final budget remains a child 5 decision) - flip infra.num_envs / infra.seed contract entries to match and add an infra.env_seed match entry guarding the new YAML field
…1456-microduck-infra-alignment
This was referenced Sep 3, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #1456 | Parent roadmap: #1452 | Base:
dev/issue-1452-microduck-rl-alignment| 依赖:#1461(已合入;本分支已同步集成分支)交付内容
env:→BackendAdapter.build_task_env_cfg_override→ManagerBasedRlEnvCfg.seed→env.rng,command/noise/delay/DR 全部消费它),缺的只是 YAML 设值。三个 microduck mjwarp owner YAML 设env.seed: 42(上游默认);默认 None→随机行为不变(mujoco owner 测试仍断言 None)。新增测试:同 seed 两次建 env 的 command 采样序列一致、不同 seed 不同。init_at_random_ep_len真实生效:上游核实 mjlab 的 wrapperepisode_length_bufsetter 直接写 env 内计数(mjlab/rl/vecenv_wrapper.py:54-59)且train.py:174确实传 True——上游错峰生效,故 UniLab 修适配层对齐而非关闭:RslRlVecEnvWrapper.episode_length_buf改 property+setter,传播进 env 新增的冷路径set_episode_length_buf(同步写episode_length_buf与state.info["steps"],形状/非负校验);env 无此方法时只更新本地记账(向后兼容)。附带收益:him_ppo/runner.py:119的同类赋值同步生效。algo.num_envs2048→4096、algo.seed1→42、max_iterations500→2000(2000×24=48000 env step 恰好覆盖全部 curriculum 终档;上游为 50000,最终预算由 microduck 对齐 child 5/5: 受控对比验收 benchmark 与统计一致性报告 #1457 对比实验决策)。infra.num_envs、infra.seed翻 match,新增infra.env_seed守卫;audit GAP 0(velocity;NOTE 剩 max_iterations 预算与 mujoco-warp 版本两个非判定项)。用户可见行为变化
microduck 三任务(ppo tree, mjwarp)默认并行规模与随机种子变化,训练 run 变为可复现(env RNG 播种后)。其他任务/后端行为不变(opt-in)。macOS/Linux 无差异。
Validation
uv run pytest tests/envs/locomotion/microduck/ -m "" -q:40 passed(含新增test_infra_alignment.py7 条:seed 复现、错峰传播后剩余长度非全同且 timeout 错峰触发)。tests/envs/ tests/algos/ -k "seed or episode_length or random_ep":12 passed;test_rsl_rl_ppo.py12 passed。uv run scripts/audit_microduck_alignment.py:MATCH 184 / GAP 0 / MISMATCH 0。make test-all:2551 passed, 28 skipped, 1 xfailed;pre-commit 与 benchmark 检查全部通过。Base 非
main,按 gate 规则使用本地验证 + review。