Add pybullet_bridge glue-construction domain#51
Merged
Conversation
This was referenced Jul 7, 2026
yichao-liang
force-pushed
the
pr6-bridge
branch
3 times, most recently
from
ec1fa7d to
b083693
Compare
New slow-process domain: build an n-shaped bridge by gluing blocks with a pickable glue bottle. Cured joints create body-to-body JOINT_FIXED welds, so the hidden cure process has a kinematic consequence: welded assemblies move as one rigid unit. Two specs: "simple" (4 blocks / 3 joints) and "full" (7 blocks / 6 joints). - envs/pybullet_bridge.py: env with per-face glue/cure/attached features, weld lifecycle (snap to yaw-only relative orientation and ideal dz to prevent constraint-vs-contact creep), visual glue patches, jittered-grid staging, both task specs. - ground_truth_models/bridge/: options (face-targeted glue dab skill, shared place skill), endogenous/exogenous processes with abstract-physical alignment guards (Loose/Resting/TopFree, single-order Attached in CureLateralJoint), NSRT mirrors, and FO/PO answer-key simulators. - skill_factories: pick gains approach_open/anchor_lift/lift_dz; place gains param_defs override and compensate_held_offset; BiRRT excludes welded partners of the held object from collision bodies. - utils: wait_option_max_steps bail-out for Waits stranded by cures that complete during the preceding option. - configs: parked bridge menu entry in envs/all.yaml plus exp_bridge.yaml launcher. - tests: oracle_process_planning E2E on the simple spec and a weld lifecycle env test. - docs/envs/bridge: init-state renders for both specs and oracle solve videos. Oracle results: simple 5/5, 5/5, 3-4/5 (seeds 0-2), PO 3/3, full 1/3 with the first task solving the complete 7-block build end-to-end.
yichao-liang
added a commit
that referenced
this pull request
Jul 7, 2026
…dation) (#52) * Fix PyBullet env leak in get_gt_options: share skill simulator/robot worlds Every get_gt_options() call leaked ~145MB of PyBullet C-heap that could never be freed (nothing calls p.disconnect): _build_skill_config built a fresh motion-planning simulator env per call when skill_phase_use_motion_planning is on, and get_options additionally ran env_cls.initialize_pybullet() per call just to obtain the SkillConfig robot handle (leaked even with the flag off). min_block_utils' _get_push_option calls get_gt_options once per K* probe, so domino min-block task generation leaked 10+GB per run and OOM'd a 16GB machine. Fix: process-wide shared_skill_simulator/shared_skill_robot caches (one world per env class) in skill_factories/base.py, used by the five factories that had the pattern (domino, boil, grow, coffee, fan). Sharing is safe because _plan_with_simulator re-syncs the full state via _set_state before every query. update_config_with_parser clears the caches on config changes (env construction reads CFG), looked up via sys.modules so processes that never use skills never load pybullet. Verified: get_gt_options x10 flat (was +145MB/call); K* probe loop byte-flat with identical K* results (was +140MB/probe); min-block task generation 2.53GB -> 0.64GB RSS. * Agent SDK: adaptive thinking replaces budget_tokens (removed on sonnet-5) With agent_sdk_model_name now claude-sonnet-5, the old thinking={type: enabled, budget_tokens: N} config is rejected with a 400 on every request (removed API surface, not just deprecated). Pass thinking={type: adaptive} at all four sandbox call sites and drop the agent_sdk_thinking_budget_tokens setting; deliberation depth is now controlled solely via agent_sdk_reasoning_effort. * Scene images: stop replans clobbering init snapshots; render explore inits _render_initial_state_image fired on every agent query -- including mid-episode replans rooted at the current, partially-executed state -- and wrote the same task{N}_initial_state.png, so post-hoc "init" images showed already-toppled dominoes. A per-episode render counter (reset in reset_for_new_episode, which CogMan never calls on replans) now names replan renders task{N}_replan{K}_state.png, and _initial_image_section references whichever file the current query actually saved, so replan prompts point at the current scene instead of the stale init. Also extract the render logic into bilevel_sketch.save_task_state_image and use it from AgentBilevelExplorer, which now saves train_task{N}_initial_state.png and injects the same "read this image first" prompt section into explore queries that test solves get. * Satisfy CI docformatter on touched files * Remove hint sentences from domino goal NL descriptions Drop the {hint} slot in _HEAVY_GOAL_NL (90-degree-turn and curve-around-obstacles hints) and the turn hint in the min-block L-task goal; also normalize an em dash to -- in the min-block goal. * Domino goal NL: allow zero blues so the believed-optimal plan is stated "Move the blue dominoes ... AS FEW as possible" presupposes moving at least one blue, which forecloses the 0-blue direct push -- exactly the plan a miscalibrated (over-reach) planner believes optimal on min-block tasks kept with believed K* < true K*. Agents therefore never validated the believed-physics optimum, dodged the differentiation trap with a conservative 1-blue bridge, and early-stopped before sysID could run. Reword all goal strings to "Arrange the blue dominoes ... (possibly none)" so under-building is the stated optimum, not a rule violation. Note: cached tasks in saved_datasets/domino_min_block_tasks embed the old goal_nl; delete them to regenerate with the new wording. * Domino: relax Toppled threshold to 30 deg to count propped leaners A domino free-stands only up to atan(depth/height) ~= 5.7 deg, so a lean past 30 deg is sustained solely by resting on another body, i.e. the domino was genuinely knocked over. The old 72 deg criterion missed propped leaners (e.g. a target at roll 0.608 rad wedged on the fallen pusher), forcing relay placements for spans a direct push handles. All call sites read comp.fallen_threshold, so the goal predicate, Tilting boundary, movable-block counting, and min-block believed-K* certification stay mutually consistent. * Add PHYSICAL_PARAMS system identification via free-running rollout fit Teacher-forced single-step fitting cannot see physical parameters of momentum-driven dynamics (states carry pose but no velocity), so friction/restitution were invisible to the existing objective. New code_sim_learning/physical_sysid.py fits agent-declared PHYSICAL_PARAMS by matching free-running base-sim rollouts of full trajectories, jointly with rule PARAM_SPECS in one posterior, and reports per-parameter identifiability (posterior contraction) instead of regularizing it away. - BaseEnv.get_physical_param_info / apply_physical_param_overrides: envs advertise tunable physics and accept sticky in-place overrides; implemented for pybullet_domino (friction, restitution, mass, rolling/spinning friction). - Synthesis prompt gains an optional PHYSICAL_PARAMS section built from the env's revealed menu; evaluate_step_fit switches to the joint rollout fit when the artifact declares PHYSICAL_PARAMS and appends the identifiability report. - Physics-only artifacts (no PROCESS_RULES) are now valid; identified values are applied to the planning base env so refinement and test planning use calibrated physics. - _solve_lm grows a diff_step override: simulation residuals need a coarser finite-difference step or the Jacobian is identically zero. - New CFG code_sim_learning_rollout_num_mcmc_steps keeps the costly rollout-fit MCMC budget separate (default 0 = LM point fit). * SysID rollouts: zero arm joint velocities between rollout evaluations resetBaseVelocity alone leaves articulated joint velocities behind, and _set_state's per-component diff skips joints whose positions already match, so consecutive rollouts on the shared fit env inherited up to ~1.8 rad/s of residual arm momentum. Position control overrides it within a step or two, so this is hygiene rather than the (engine internal) source of the observed same-theta SSE jitter, but rollouts genuinely must start at rest. * SysID identifiability: discount probe curvature below the noise floor The prior-scale curvature probe measured the SSE increase against a single MAP evaluation, so on chaotic data the same-theta evaluation jitter itself read as curvature and every parameter was declared identified (run_20260705_203314: ~5k d_sse inside a ~8k jitter, contraction 0.00 on all params, false confidence in friction 0.5). Evaluate the MAP SSE three times, take the median, and subtract the observed spread from every perturbation response before it counts as curvature; a flat-after-discount direction reports NOT identified. * SysID fit: truncate settled trajectory tails and grid-seed the LM start Two measured blockers on real domino data (true friction 0.1, planning 0.5): - A free-running rollout diverges chaotically from the recording over hundreds of contact steps, and the long settled tail re-scores that accumulated divergence every step: on full 500-step trajectories the SSE at the TRUE friction exceeded the SSE at the wrong one. Cutting each trajectory after its last observed scored-feature motion (plus a settle margin) restores the signal: a clean top-edge push goes from ratio 0.9 to ~5e4 wrong/true SSE. - The SSE landscape is flat above friction ~0.5 (topple reach saturates), so LM's finite differences see no gradient from the declared init and the fit stalls even on clean data. A coarse per-parameter grid sweep relocates the LM starting point into the best-scoring basin; the Gaussian prior stays centered on the declared init. Together these recover friction 0.07-0.08 from a single clean recorded push (init 0.5); previously the fit returned the init unchanged. * SysID robustness: trim untrustworthy trajectories, guard param application The fit consumed whatever exploration recorded, and applied whatever it fitted. Both trusted too much (run_20260706_111805: a chaotic scraping push dragged the pooled friction fit to 0.34, and rolling/spinning friction were applied at arbitrary values their own identifiability report called NOT identified). Three layers, each verified against real recorded episodes: - Explainability trimming (min_explainable_rms): judge each trajectory by its best-achievable RMS over a candidate param grid — judged against its own best params, not the pooled fit, which chaos poisons — and drop recordings no parameters can explain before fitting. - Consistency loop: at the joint fit every survivor must fit nearly as well as its own best; on disagreement (a recording can be accidentally explainable at WRONG params, e.g. a quiet shove at friction ~1.34 vs the clean topple's 0.1) drop the survivor with the largest best-RMS and refit, anchoring on the cleanest data. - select_trustworthy_params: apply a fitted physical value to the planning env only when its identifiability verdict is at least weakly identified; keep the declared init otherwise. If nothing survives trimming, the result is pinned at the declared inits so a false-positive verdict on chaotic data cannot leak arbitrary values. Callers compute post-SSE and the identifiability probe on the SURVIVING trajectories only. E2E on real recordings: chaotic-only data keeps the declared inits with honest NOT-identified verdicts; chaotic pooled with one clean topple trims the chaos and recovers friction 0.0706 (true 0.1) plus rolling_friction 0.0078 (true 0.006). * sysID: fit scale-like physical params in log-space run_20260706_171526 fit friction 0.0114 for a true 0.1: the linear grid over [0.01, 2.0] has no candidate between 0.01 and 0.29, the sweep jumped to the 0.01 endpoint, LM saw no gradient across the flat low-friction basin, and the linear curvature probe blessed the bound-hugging MAP as identified. The believed sim ended up 9x too slippery and drop-place plans verified in sim but failed in the real env (the 2/5 final-test failures). ParamSpec gains scale="log" for positive scale-like parameters. The optimizer runs in z = log(theta): geometric grid sweeps (a candidate lands at ~0.098), relative LM finite-difference steps, a log-normal prior (x2 up and x2 down equally plausible), log-space identifiability contraction, and fit-space Laplace/perturbation ensembles. Everything simulator- and caller-facing stays in linear units; FitResult samples stay external while jacobian/prior_sigma live in fit space (scales records the mapping). The domino env registry marks friction, spinning_friction, and mass as log-scale (restitution and rolling_friction keep lo=0, stay linear), and the approach/tool paths stamp the registry scale onto the agent-declared PHYSICAL_PARAMS so agents need not know about it. E2E-verified on a same-engine cascade at true friction 0.1 from init 0.5: log-scale recovers 0.0987 (SSE 0.705 -> 0.0005); the old linear path reproduces the 0.0100 endpoint failure on identical data. * Satisfy CI checks (autoformatters + pylint docstrings) on touched files * Fan: run the agent grid-free via oracle-injected grid helpers Mirror the domino helper-object design so the fan agent plans over a physical-only vocabulary while the loc/side grid is injected for the oracle / process-planning approaches only. - New ground_truth_models/fan/{types,predicates}.py: the loc/side helper types and the grid predicates (BallAtLoc/ClearLoc/SideOf/FanFacingSide/ OppositeFan), plus augment_{task,state}_with_helper_objects that rebuild the exact task grid (coords encoded in loc names) and rewrite the goal BallAtTarget -> BallAtLoc. - pybullet_fan.py: drop loc/side from types and the grid predicates from the env; add the physical BallAtTarget goal predicate; surface the target coordinates through per-task goal_nl; reconstruct injected loc/side features from object names. Two-table workspace with y_ub=2.1 and front-anchored robot/switches so the arm never reaches into the grid. - ground_truth_models/__init__.py: add the augment_state_with_helper_objects hook (base no-op + dispatcher). - process_planning_approach.py: re-derive helper objects on every execution state before abstracting, so the closed-loop oracle keeps evaluating BallAtLoc during execution. No-op when helpers are disabled. * Fan GT hybrid simulator: model the wind that base sims skip Unlike domino (plain rigid-body toppling, no-op GT sim), fan applies its wind in _domain_specific_step, which skip_process_dynamics=True base sims never run - so the GT simulator must reproduce the ball's wind-driven motion itself. Calibrated against the real env: constant 0.00228 m/action ball speed, zero coasting, sphere-overhang wall parking, boundary clamp from the target-inferred grid. Params ball_speed + wall_clearance (hard hinge clamps keep the LM Jacobian informative). Unblocks the agent_oracle_hybrid_sim arm; lockstep hybrid-vs-real tests stay within 14.6mm << the 40mm goal tolerance. * Satisfy CI checks (autoformatters + pylint) on touched files * Default code_sim_learning_num_mcmc_steps to 0; prune redundant agents.yaml flags MCMC was already disabled via --code_sim_learning_num_mcmc_steps 0 in every experiment config; make that the default (LM point fit only). Drop the five flags in the active agent_po_predicate_invention_al arm that now just restate settings defaults (option_model_use_gui, agent_bilevel_log_state, agent_sim_learn_oracle_sim_program, agent_sim_learn_oracle_sim_params, code_sim_learning_num_mcmc_steps). agents.yaml also carries the arm toggling from the sysid work in progress: agent_po_predicate_invention_al re-activated, the agent_base_sim_no_learning / agent_oracle_hybrid_sim arms commented out. * Configs: unpark friction domino env for the sysID experiment; keep videos Swap the active min-block arm back from the heavy-block variant to the friction-mismatch domino env, and enable failure/test videos in common launch args for run inspection. * predicatorv3 configs: thin launchers over a canonical approaches menu Extract every approach ("arm") into approaches/all.yaml, parked with SKIP: True, and turn the launchers into thin includes that only un-skip the arm(s) they run plus per-experiment ENVS overrides: - oracle.yaml now includes approaches/all.yaml and flips oracle SKIP: False - agents.yaml -> exp.yaml (the "ours" arm launcher) - predicator_v3.yaml removed (its approach defs now live in the menu) * predicatorv3 envs: enable the fan env; park the domino sysID env Un-comment the pybullet_fan block in envs/all.yaml so the fan oracle runs on this branch, and set the friction domino env SKIP: True (reversing the sysID unpark) so only fan runs here. * Add shared use_gt_helpers flag for both planning families Rename the process-planning-only process_planning_use_gt_helpers to a general CFG.use_gt_helpers read by both the process-planning and the agent-planning approaches, so an "agent-with-grid" ablation can hand the LLM agent the same oracle scaffolding (domino/fan grid loc/side types + grid predicates). Default False; the oracle still hard-overrides to True. - settings.py + 3 ExoPredicator configs: rename the flag. - ground_truth_models: extract merge_gt_helper_types / merge_gt_helper_predicates (name-collision precedence) as the single definition of the helper-vocab merge; process_planning_approach now calls them. - agent_planner_approach (base of every agent arm): when use_gt_helpers, merge helper types/predicates into the vocabulary before the agent session is built, augment the solved task with the grid + oracle goal, and re-derive the grid on every execution state for the policy's and the option model's abstraction (Wait-on-atom-change). Exploration-time augmentation is left as a follow-up. - fan/domino augment_task_with_helper_objects: preserve goal_nl (the agent needs the NL goal even though the symbolic goal becomes BallAtLoc). * predicatorv3: split per-experiment launchers exp_domino / exp_fan Env selection now mirrors the approaches menu: every env in envs/all.yaml is parked with SKIP: True and each thin launcher un-skips exactly the env + arm it runs. exp_domino.yaml reproduces the old agents.yaml run (domino friction-sysID env x agent_po_predicate_invention_al) and exp_fan.yaml reproduces the fan branch's exp.yaml run (fan env x agent_oracle_hybrid_sim); both verified flag-for-flag identical to their predecessors via generate_run_configs. oracle.yaml keeps running the fan oracle and now says so explicitly. * Add pybullet_bridge glue-construction domain (#51) * Add pybullet_bridge glue-construction domain New slow-process domain: build an n-shaped bridge by gluing blocks with a pickable glue bottle. Cured joints create body-to-body JOINT_FIXED welds, so the hidden cure process has a kinematic consequence: welded assemblies move as one rigid unit. Two specs: "simple" (4 blocks / 3 joints) and "full" (7 blocks / 6 joints). - envs/pybullet_bridge.py: env with per-face glue/cure/attached features, weld lifecycle (snap to yaw-only relative orientation and ideal dz to prevent constraint-vs-contact creep), visual glue patches, jittered-grid staging, both task specs. - ground_truth_models/bridge/: options (face-targeted glue dab skill, shared place skill), endogenous/exogenous processes with abstract-physical alignment guards (Loose/Resting/TopFree, single-order Attached in CureLateralJoint), NSRT mirrors, and FO/PO answer-key simulators. - skill_factories: pick gains approach_open/anchor_lift/lift_dz; place gains param_defs override and compensate_held_offset; BiRRT excludes welded partners of the held object from collision bodies. - utils: wait_option_max_steps bail-out for Waits stranded by cures that complete during the preceding option. - configs: parked bridge menu entry in envs/all.yaml plus exp_bridge.yaml launcher. - tests: oracle_process_planning E2E on the simple spec and a weld lifecycle env test. - docs/envs/bridge: init-state renders for both specs and oracle solve videos. Oracle results: simple 5/5, 5/5, 3-4/5 (seeds 0-2), PO 3/3, full 1/3 with the first task solving the complete 7-block build end-to-end. * Satisfy CI checks (autoformatters + mypy + pylint) on touched files
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contents
envs/pybullet_bridge.py+ground_truth_models/bridge/{gt_simulator,gt_simulator_po,nsrts,options,processes}.pyskill_factories/{base,pick,place}.pyexp_bridge.yamllauncher,docs/envs/bridge/(README, init snapshots, oracle solve videos)tests/envs/test_pybullet_bridge.py,tests/approaches/test_oracle_process_planning_bridge.pyMerges last.