Skip to content

AI assistant v3: flow-node behaviors + physics tools - #76

Merged
AlexZ005 merged 2 commits into
release/nextfrom
feature/ai-flow-physics-tools
Aug 1, 2026
Merged

AI assistant v3: flow-node behaviors + physics tools#76
AlexZ005 merged 2 commits into
release/nextfrom
feature/ai-flow-physics-tools

Conversation

@AlexZ005

@AlexZ005 AlexZ005 commented Aug 1, 2026

Copy link
Copy Markdown
Collaborator

The assistant can now create behavior, not just objects: flow nodes always, and physics (bodies, joints, simulation control) behind a per-provider opt-in. Target prompt: "create a moving spider" — body + legs from primitives, grouped, patrolling the scene with bouncing legs, simulated.

What's in

  • 'flownodes' history kind (flowGraphs.js) + recordFlowNodesEntry: node/edge create, data edits and removals as one undoable entry; entries carry SERIALIZED copies so replayed re-broadcasts hash identically on every peer (nodesync drift guard).
  • setPhysicsFor(uuid, patch) (physics.js): the shared physics write path — props history + objectParameters replication + collider-viz poke + the CL-A A2 live mid-sim collider rebuild. Inspector setPhysics and enablePhysicsOnSelection refactored onto it, so all three callers (Inspector / quick action / AI) behave identically.
  • src/lib/ai/flowTools.js (new): the five executors. create_flow_nodes / update_flow_nodes are always available (curated node-type enum + alias map, editor-identical node/edge construction incl. the handle-qualified edge-id format, implicit-owner leverage: one node + zero edges drives the graph's object). set_physics / create_joints / control_simulation gate behind the provider's Physics tools checkbox — as do the physics-family node types (mass/bounciness/friction/angularvelocity/motor + the CL-C collider override, onimpact/onenter/onexit triggers, velocity readout).
  • tools.js wiring: schemas (built per call so the enum tracks the gate), dispatch, repairToolCall additions (5 names, invention aliases like add_behavior/start_simulation with action fill-in, shape inference incl. physics-only updatesset_physics), summarizeScene now carries per-object physics + compact flow (12-node cap) plus top-level sceneFlow + joints; system prompt gains the flow block (spider recipe) and a gated physics block (world-aligned jointing, auto sim start).
  • assistant.js labels + tally (control_simulation counts 0); providers.js physicsTools flag; Settings → AI checkbox with a link to the new docs page.
  • Docs: ai/local-models.md (qwen3_xml-not-hermes vLLM guidance, model-size table, checkbox semantics) already committed on the theprototype-docs repo.

Semantics worth noting

  • One prompt = one undo, but sim start/stop is NOT part of the batch — undo never stops a running simulation (stop/reset first, then one undo reverts objects + nodes + physics + joints together).
  • set_physics supports the CL-A A3 sensor flag; custom colliders and freeze axes stay editor-only.

Verification

  • New suite tests/e2e/ai-flow-physics.test.cjs — 38 checks over a scripted mock endpoint: the full spider scenario (create → group → pathpatrol on the GROUP graph + bounce on a leg via refs → wired scene-graph pair → update_flow_nodes speed tune → static ground → hinge → sim start), broadcast capture, gating (disabled error + per-item physics-node rejection), invented-name repair, one-undo/redo-without-sim.
  • Regressions green post-merge: ai-assistant, ai-local-model, flow-physics-collider, physics-joints, flow-object-graphs.
  • npm run build green; npx svelte-check 438/62 — exactly the release.yml baseline gate.
  • Merged release/next in (colliders v2 conflict resolved by moving physicsShapeChanged into setPhysicsFor).
  • Manual check remaining: live smoke against the real vLLM box (--tool-call-parser qwen3_xml) — "create a moving spider" / "make the spider faster".

🤖 Generated with Claude Code

AlexZ005 and others added 2 commits July 29, 2026 00:13
- flowGraphs.js: new 'flownodes' history kind (create/delete/data ops, one
  undoable entry per AI call; serialized copies so re-broadcasts hash
  identically) + recordFlowNodesEntry
- physics.js: setPhysicsFor(uuid, patch) shared write path (props history +
  objectParameters replication); enablePhysicsOnSelection and Inspector
  setPhysics refactored onto it
- new src/lib/ai/flowTools.js: create_flow_nodes / update_flow_nodes (always
  available) + set_physics / create_joints / control_simulation (gated by a
  per-provider Physics tools checkbox); curated node-type enum with alias map,
  editor-identical node/edge construction, implicit-owner leverage
- tools.js: schemas + dispatch + repairToolCall (5 new names, invention
  aliases, shape inference incl. physics-only updates -> set_physics),
  summarizeScene now carries per-object physics + compact flow, top-level
  sceneFlow + joints; system prompt gains flow block + gated physics block
  (spider recipe, world-aligned jointing, auto sim start)
- assistant.js: status labels + tally for the new tools (control_simulation
  counts 0); providers.js physicsTools flag; Settings -> AI checkbox with
  docs link
- e2e: ai-flow-physics.test.cjs (38 checks) - scripted moving-spider scenario
  over a mocked endpoint, gating + per-item physics-node rejection, invented
  add_behavior repair, one-undo/redo semantics, broadcast capture
- verified: build green, svelte-check 437/62 (baseline held), suites
  ai-assistant ai-local-model flow-object-graphs physics-joints all pass

Co-Authored-By: Claude <noreply@anthropic.com>
…inors) into ai-flow-physics-tools

- Inspector setPhysics stays on the shared setPhysicsFor path; the CL-A A2
  live mid-sim collider rebuild (physicsShapeChanged) + collider-viz poke
  move INTO setPhysicsFor so the AI set_physics tool and the quick action
  get them too
- flowTools PHYSICS_NODE_TYPES extended to the CL-C physics family
  (collider override node + onimpact/onenter/onexit triggers + velocity
  readout) so they gate with the physics checkbox like mass/motor
- set_physics gains the CL-A A3 sensor flag (trigger volumes)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AlexZ005
AlexZ005 merged commit 2983fff into release/next Aug 1, 2026
AlexZ005 added a commit that referenced this pull request Aug 1, 2026
…ture skill

- CLAUDE.md status: replace the not-yet-merged note with the PR #76 summary
  (flow tools always on, physics tool set + physics node FAMILY behind the
  per-provider checkbox, setPhysicsFor as the shared physics write path now
  carrying the CL-A A2 live rebuild, summarizeScene flow/physics/joints,
  repair additions, sim start/stop outside the undo batch, post-merge
  verification state)
- CLAUDE.md map: flowGraphs.js entry notes the 'flownodes' history kind
  (serialized copies so replayed re-broadcasts hash identically)
- peer-feature skill: history entries that re-broadcast hash-checked content
  must store serialized copies, never live editor objects ('flownodes' is
  the reference)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@AlexZ005
AlexZ005 deleted the feature/ai-flow-physics-tools branch August 5, 2026 20:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant