Realize editor scripting through Python

[markaren]

class Spinner:
    speed = 1.5          # shows up in the inspector, saved with the scene

    def start(self, obj):
        self.obj = obj

    def update(self, dt):
        self.obj.rotation.y += self.speed * dt

    def stop(self):
        pass

Builds on #392

[markaren]

Status: realized ✅

Editor scripting through Python is no longer a plan — it shipped (#394, #396, #397), and it goes considerably deeper than this issue originally asked for.

Two kinds of scripts

Behaviour scripts (MonoBehaviour-style): attach a Python class to any object — a .py file shared between scenes, or inline source stored in the document itself. Press Play and it runs; press Stop and the scene snaps back. Lifecycle: start(obj), update(dt), fixed_update(dt) (runs on the physics clock via a pre-substep callback, so controllers are frame-rate independent), stop(), plus on_collision_enter/exit(contact) and on_trigger_enter/exit(other) — contact reporting is enabled automatically, and a touch that begins and ends within one frame still delivers enter then exit.

Generator scripts: a scene can carry inline Python that builds it. The script authors into a detached sink via threepp.editor.add(), and the finished subtree is committed as one undoable step — a script that raises halfway commits nothing. Seeded, and the committed output ships in the document, so opening a scene never executes code.

The threepp.editor API

Scripts import the same threepp module the wheel provides (see architecture below), plus an editor-only submodule:

• scene(), add() — authoring and play-scene access
• world(), rigid_body_from_object(), soft_body_from_object(), articulation_from_object() — lifetime-checked handles onto the bodies the play session is already simulating (forces, impulses, kinematic targets, PD drive targets per joint)
• raycast() against the playing world
• imu_from_object(), encoder_from_object(), contact_from_object(), force_torque_from_object() — read the authored, noisy, seeded sensors, not ground truth, with latest() / read_new() semantics
• spline_from_object() — arc-length point/tangent sampling of authored splines
• script_from_object() — one script reaching another's live instance; guaranteed to work regardless of scene order (every instance exists before any start() runs)
• is_key_down() — driveable scenes; answers False headless or while the user is typing in a text field
• draw_line/ray/point/axes — immediate-mode debug draw into the viewport overlay (capped, headless-safe)

Plain class attributes (bool/int/float/str) become inspector-editable parameters with per-object overrides stored in userData — round-trippable even by builds without Python.

Editing: docked panel + VS Code

• The Script Editor docks into the bottom panel, one tab per open script, with syntax checking on Apply (compile only — nothing runs before Play) and undoable commits.
• "Edit in VS Code" works for both file and inline scripts. It generates a .vscode/settings.json pointing Pylance at the threepp type stubs, so import threepp completes with full IntelliSense, and opens the folder + file so the workspace settings actually apply. Inline scripts are exported to a scratch .py and polled once a second: every save round-trips through the same normalization, syntax check, and undo transaction a typed edit gets. Saves during Play are parked and applied on Stop; saving a generator auto-triggers Regenerate — a real edit-save-look loop.

Architecture worth bragging about

• Embedded interpreter via PYBIND11_EMBEDDED_MODULE that links the exact same binding TUs the wheel is built from — one type registry, zero binding drift between the editor and the threepp Python package.
• threepp.PhysxWorld() construction is denied with an error message that explains what to call instead (threepp.editor.world()) — the session owns the one world.
• Error containment: a raising script is logged once, disabled for the session, and everything else keeps playing; errors are kept per-object for the inspector after Stop.
• Object handles are concrete-typed (Mesh, Robot, …) and shared_ptr-held, so a script stashing one across Stop is safe.
• Python is an optional dependency: without it the editor still builds, authors script configs, and round-trips parameters.

Beyond the editor

• threepp_player (doc/player.md): a headless-capable player that runs the same ScriptPlaySession — a CI gate for authored scenes. Any script raise → exit code 1. Windowed runs feed the keyboard to scripts too.
• Hover Arena, a shipped example using the whole runtime: a drone holding hover from fixed_update forces closed on a noisy authored IMU, trigger-ring scoring via script_from_object, and the arena itself built by a seeded generator.
• A trained RL policy runs inside the editor through this API (python/examples/spot/editor_deploy/spot_editor_script.py) — attach to a URDF robot, press Play, the policy drives it.