agent_sdk: session-probe tool surface, sysID subset fits, solve-journal lifecycle (sysID stack 1/8) - #96
Merged
Merged
Conversation
… stale arms
agent_oracle_hybrid_sim now carries the full long_python config (200-turn
budget, 900px images, explore_python, solve restart loop), so remove the
superseded agent_oracle_hybrid_sim_{max,long,long_python,hires} arms and
the stale domino_turns_new_goal/domino_900/domino_infront env variants.
exp_domino.yaml runs domino_turns x agent_oracle_hybrid_sim; delete the
exp_domino_hybrid_arms.yaml launcher (referenced an undefined approach).
…lve restart loop Match the oracle hybrid-sim arm's session setup on the PO invention arm: the explore_python probe (its only solve-time visualization/refinement surface), the 200-turn session budget, capped image resolution, and the time-boxed fresh-context solve attempts with the cross-attempt journal.
…he session probe
One exec surface per session: solve keeps explore_python, synthesis
keeps run_python with the ProbeSim facade merged into its namespace
(unconditionally - the probe is the only validation surface).
- inspect_types/inspect_options digests are prompt-injected via shared
renderers; inspect_trajectories/inspect_train_tasks subsumed by the
probe namespace (trajectories, describe_trajectory, sim.task).
- evaluate_step_fit -> sim.fit (no-arg = canonical fit, applies
identified sysID params; traj_idxs/fixed = exploratory on the
rule-param paths, publishing nothing).
- evaluate_plan_refinement deleted: validation is hand-composed as
sim.fit + sim.refine + one continuous sim.run, whose per-step
'-> {subgoals}' checks are now reported (SUBGOAL NOT REACHED lines;
previously computed in execute_plan_forward but dropped).
- report_residuals -> sim.residuals (same math, same version tags via
the shared session snapshotter).
- annotate_scene/visualize_state removed entirely (tools, CFG flags,
ctx.visualized_state); sim.reset(mods) + sim.render(annotations)
are the visualization surface, scene.py helpers stay as its backend.
- run_refinement_for_synthesis/get_or_build_sketch deleted;
build_candidate_option_model remains the single fit-publish path.
- Rosters, prompts, and blurbs updated everywhere; standalone sampler
session gets the probe + fit/residual providers (deployed model).
- SYNTHESIS_TOOL_NAMES = (run_python,); scene flags stripped from
approaches/all.yaml.
…it(traj_idxs) On the PHYSICAL_PARAMS rollout path, sim.fit(traj_idxs=[...]) now runs the full sysID stack on just those trajectories as an EXPLORATORY cross-trajectory consistency diagnostic: the trimmer only drops per-segment unexplainable chaos, so it cannot see trajectories that are individually explainable but pull a parameter in different directions (the replay-divergence-bias failure class) - disagreeing subset fits expose exactly that. Nothing is applied or recorded; only the no-arg canonical fit can touch the deployed params. - Subsetting happens on the SOURCE trajectory list (before completeness filtering, truncation, and rest-point segmentation) so indices match the session's trajectories list; out-of-range and empty subsets are refused with tagged errors. - Exploratory fits use a private explainability cache: the phase-shared cache keys trajectory identity by segment lengths only, so two different subsets with equal-length segments would collide and cross-assign explainability verdicts. - fixed stays rejected on this path: pinning a physical param goes through its lo/hi bounds in the PHYSICAL_PARAMS declaration (the versioned channel the whole fit stack respects).
SimpleNamespace stubs assigned to _fit_trajectories (typed List[LowLevelTrajectory]) failed mypy --platform linux; cast them explicitly. Pre-dates this branch's changes (reproduces at fb2c609).
…ts, eval rolls back)
Learning-phase journal entries persist across online-learning cycles and
are visible during every evaluation. Each evaluation starts from the
learning entries only: begin_test_phase snapshots journal.md
(journal.read_raw), and end_test_phase archives the full journal
(learning + this eval's entries) to <log_dir>/journal_eval_cycle{C}.md -
one copy per cycle, outside the sandbox so the agent cannot read it -
then rolls the file back to the snapshot (journal.restore), so one
eval's test-task entries never leak into the next eval.
The model-based approach also rolls back the per-task context-entry
dedup keys added during the phase, so a later eval re-writes its goal +
init entries. A failed snapshot skips the rollback instead of deleting
learning entries. _journal_active moves up to AgentModelFreeApproach.
This was referenced Jul 30, 2026
code_sim_learning: sysID verdicts, shared orchestrator, anchor-ablation refits (sysID stack 3/8)
#98
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First of eight stacked-in-sequence PRs landing the domino-sysid branch (53 commits) on master in reviewable chunks. Each PR is a contiguous slice of that history; later PRs build on earlier ones.
This chunk
Consolidates the agent tool surface into one exec namespace per session (inspect digests prompt-injected, probe available inside run_python), adds exploratory subset fits for sysID (
sim.fit(traj_idxs)), and gives the solve journal a phase lifecycle so learning-phase entries persist while eval-phase entries roll back.🤖 Generated with Claude Code