Infra: PyBullet env-leak fix, adaptive-thinking SDK, scene-snapshot guard#46
Merged
Conversation
…worlds Every get_gt_options() call leaked ~145MB of PyBullet C-heap that could never be freed (nothing calls p.disconnect): _build_skill_config built a fresh motion-planning simulator env per call when skill_phase_use_motion_planning is on, and get_options additionally ran env_cls.initialize_pybullet() per call just to obtain the SkillConfig robot handle (leaked even with the flag off). min_block_utils' _get_push_option calls get_gt_options once per K* probe, so domino min-block task generation leaked 10+GB per run and OOM'd a 16GB machine. Fix: process-wide shared_skill_simulator/shared_skill_robot caches (one world per env class) in skill_factories/base.py, used by the five factories that had the pattern (domino, boil, grow, coffee, fan). Sharing is safe because _plan_with_simulator re-syncs the full state via _set_state before every query. update_config_with_parser clears the caches on config changes (env construction reads CFG), looked up via sys.modules so processes that never use skills never load pybullet. Verified: get_gt_options x10 flat (was +145MB/call); K* probe loop byte-flat with identical K* results (was +140MB/probe); min-block task generation 2.53GB -> 0.64GB RSS.
…t-5)
With agent_sdk_model_name now claude-sonnet-5, the old
thinking={type: enabled, budget_tokens: N} config is rejected with a 400
on every request (removed API surface, not just deprecated). Pass
thinking={type: adaptive} at all four sandbox call sites and drop the
agent_sdk_thinking_budget_tokens setting; deliberation depth is now
controlled solely via agent_sdk_reasoning_effort.
…inits
_render_initial_state_image fired on every agent query -- including
mid-episode replans rooted at the current, partially-executed state --
and wrote the same task{N}_initial_state.png, so post-hoc "init" images
showed already-toppled dominoes. A per-episode render counter (reset in
reset_for_new_episode, which CogMan never calls on replans) now names
replan renders task{N}_replan{K}_state.png, and _initial_image_section
references whichever file the current query actually saved, so replan
prompts point at the current scene instead of the stale init.
Also extract the render logic into bilevel_sketch.save_task_state_image
and use it from AgentBilevelExplorer, which now saves
train_task{N}_initial_state.png and injects the same "read this image
first" prompt section into explore queries that test solves get.
This was referenced Jul 7, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commits
get_gt_options-get_gt_optionswas spinning up (and leaking) two PyBullet worlds per call (a sim world + a robot-init world). Share them throughshared_skill_simulator/shared_skill_robotcaches inskill_factories/base.py. Root cause of ~12GB OOM growth over a run.budget_tokens-budget_tokenswas removed on sonnet-5; switch the sandbox/session runners to adaptive thinking driven byagent_sdk_reasoning_effort.Base of the stack; targets
master.