Skip to content

Release 2.0.0

Choose a tag to compare

View all tags
@AGulev AGulev released this 18 Jul 08:34
· 30 commits to master since this release
2.0.0
807e079
This commit was created on GitHub.com and signed with GitHub’s verified signature.
GPG key ID: B5690EEEBB952194
Verified
Learn about vigilant mode.

Highlights

  • Increased the minimum supported Defold version from 1.12.4 to 1.13.1.
  • Added deterministic FIFO input execution with lifecycle receipts and safe cancellation.
  • Added an opt-in Lua synchronization API for events, state, commands, input acknowledgements, and semantic annotations.
  • Added snapshot-safe scene queries, coordinate conversion, frame correlation, and atomic screenshots.
  • Added native H.264 MP4 recording on macOS and Windows without FFmpeg.
  • Added Metal .gputrace capture on macOS.
  • Redesigned the Python wrapper around explicit editor and engine namespaces.
  • Added editor discovery, build workflows, preview rendering, console access, debugger control, references, and typed preferences.
  • Added structured traces, visual comparisons, generated gestures, bounded runtime logs, and improved profiling.

Breaking Python API changes

  • The package root now exports only automation_bridge.editor and automation_bridge.engine.
  • The Python package version is now 2.0.0; the native extension reports 1.3.0. The HTTP API remains v1.
  • Replaced AutomationBridgeClient.from_project(...) with:
    • editor.open_project(...)
    • project.build_and_run()
    • project.clean_build_and_run()
    • project.connect_engine()
    • engine.connect(port)
  • Renamed public scene terminology from Node to Element.
  • Renamed:
    • nodes() → elements()
    • node() → element()
    • maybe_node() → maybe_element()
    • by_id() → element_by_id()
    • wait_for_node() → wait_for_element()
    • observe_node() → observe_element()
    • format_nodes() → format_elements()
    • wait_for_ack() → wait_for_input_acknowledgement()
    • recording → video_recording
  • Removed compatibility aliases and direct get(), post(), put(), delete(), post_json(), and put_json() helpers. Use request(...) for raw endpoint access.
  • Removed optional; use supports(...) for optional capabilities and require(...) for mandatory capabilities.
  • Moved pixel operations under game.visual, gesture generation under game.gestures, recording under game.video_recording, Metal capture under game.metal_capture, and profiling under game.profiler.
  • click(), drag(), and drag_path() wait for native release by default.
  • type_text() and key() return after acceptance by default unless another wait state is requested.
  • Public input helpers return typed native receipts instead of estimating completion with sleeps.

Deterministic input and pointer control

  • Serialized clicks, drags, drag paths, pointers, and keys through one native FIFO.
  • Added controller ownership using client_id, session_id, request_id, and renewable leases.
  • Added lifecycle receipts with:
    • accepted, started, released, cancelled, and failed states
    • queue position
    • accepted/start/release frames and timestamps
    • requested and actual duration
    • device and pointer identity
    • engine instance, scene sequence, and request correlation
  • Added input status, pending-queue inspection, cancellation, session flushing, and controller configuration.
  • Added safe release of held mouse buttons, touch contacts, and keys when an action is cancelled, a lease expires, or a script is interrupted.
  • Added explicit auto, mouse, and touch device selection without injecting the same gesture through multiple devices.
  • Added capability reporting for supported input devices.
  • Added stale-scene protection through expected_scene_sequence for node-targeted input.
  • Added continuous drag paths with:
    • sampled and linear segments
    • quadratic and cubic Bézier curves
    • per-segment durations and easing
    • pre- and post-gesture holds
    • 2–128 control points
    • visualization of the actual injected path
  • Added leased low-level pointer sessions with open, move, hold, up, and cancellation operations.
  • Added structured InputReceipt, InputController, PointerSession, and interruption-scope APIs in Python.
  • Added bounded structured JSON bodies for input mutations to prevent long query strings from being truncated.

Application synchronization

  • Added the debug-only automation_bridge Lua module.

  • The application API is opt-in through:

    [automation_bridge]
application_api = 1
event_capacity = 256

  • Added structured application events with:

    • monotonic sequence numbers
    • explicit cursors
    • bounded retention
    • long polling
    • overflow detection without silently skipping lost events

  • Added revisioned published state with exact-name lookup and long polling for newer revisions.

  • Added registered application commands with:

    • strict JSON arguments
    • command IDs
    • pending, running, completed, failed, cancelled, and timed-out states
    • result and error records
    • cancellation of pending commands

  • Added explicit application acknowledgement of native input through automation_bridge.acknowledge_input(...).

  • Added semantic scene annotations through automation_bridge.annotate(...):

    • automation_id
    • localization_key
    • role

  • Added timeline markers with native monotonic timestamps and optional recording-clock correlation.

  • Added typed Python Event, EventStream, StateSnapshot, overflow, and command-timeout APIs.

  • Event streams resolve "now" before an action begins, preserve unmatched events, and report buffer overflow explicitly.

  • Added Python helpers for events, state waits, commands, markers, and full input acknowledgements.

Scene inspection and synchronization

  • Added scene_sequence and engine_frame correlation to scene, node, screen, input, screenshot, and observation results.
  • Added snapshot IDs distinct from persistent engine instance identity.
  • Added game-object instance identifiers, allocation generations, logical IDs, and first-observed scene sequences.
  • Fixed unsafe collection-root identity handling that could dereference uninitialized instance data during health discovery.
  • Added server-side selectors for:
    • exact type, name, text, and URL
    • path, kind, instance ID, and logical ID
    • semantic automation ID, localization key, and role
    • visible, enabled, bounded, and visible-and-enabled state
  • Added case-sensitive substring matching when requested.
  • Added pagination with offsets/cursors, truncation state, next cursors, and complete match counts.
  • Added active-collection and selector diagnostics.
  • Added stale-snapshot rejection before resolving input targets.
  • Zero is now accepted as a valid stale scene-sequence sentinel and returns stale_scene instead of bad_request.
  • Added Python element appearance, disappearance, count, and multi-frame observation helpers.
  • Added clearer selector errors with nearby candidate diagnostics.

Coordinates, frames, and resizing

  • Defined top-left coordinate spaces for:
    • window
    • client area
    • drawable display pixels
    • backbuffer
    • viewport
    • normalized viewport
    • configured project content
  • Added /coordinates/convert for conversions between advertised native spaces.
  • Added /frame for native engine-frame and scene-sequence observation.
  • Expanded screen metadata with named rectangles, offsets, project scale, scene sequence, and engine frame.
  • Resize operations now distinguish:
    • already correct
    • successfully resized
    • resize requested but not yet observed
    • unsupported resize
  • Added portrait and landscape helpers that retain the last known window size.
  • Added frame waits with timeouts calculated from the requested frame count.
  • Frame-wait failures now report initial and final frames, whether frames advanced, lifecycle stage, and best-effort health data.

Atomic screenshots and visual checks

  • Replaced screenshot file-size polling with retained capture IDs and explicit pending, complete, and failed states.
  • Added post-render after_frames scheduling.
  • Added frame, scene-sequence, dimensions, path, failure reason, and SHA-256 metadata.
  • Screenshots are written to temporary files and atomically renamed before completion is published.
  • Standardized screenshot rows, scene bounds, pointer input, and coordinate conversion on top-left orientation.
  • Fixed vertically inverted screenshots on macOS by preserving Defold’s already normalized readback row order.
  • Added runtime screenshot scaling from 0.01 through 1.0, using bilinear downscaling while retaining the native source path.
  • Added dependency-free PNG comparison with normalized mean absolute error.
  • Added:
    • region comparisons
    • expected-image assertions
    • stable-frame waits
    • region-change waits
    • typed visual observation results

Native video recording

  • Added /recording/capabilities, /recording/status, /recording/start, and /recording/stop.
  • Recording runs inside the Defold process and does not require FFmpeg or another external recorder.
  • Added native H.264 MP4 recording on macOS 15+ using ScreenCaptureKit.
  • macOS recording:
    • automatically selects the Defold process window
    • captures the undecorated game content
    • supports application audio
    • supports resizing and 1–60 FPS
    • excludes title bars, shadows, and other applications
    • removes the rounded bottom-window safety area to produce rectangular video
  • Added native H.264 MP4 recording on Windows 10 version 1903+ using Windows Graphics Capture, Direct3D 11, and Media Foundation.
  • Windows recording:
    • selects the largest visible Defold-owned window
    • crops to the Win32 client area
    • supports resizing and FPS control
    • temporarily disables Windows 11 rounded corners and restores them afterward
    • is video-only and explicitly rejects audio=true
  • Added synchronous finalization and output validation.
  • Fixed the Windows frame-callback lifetime and shutdown sequence using an agile WinRT handler.
  • Added typed recording capabilities, metadata, error, client, and context-managed session APIs.
  • Recording finalization preserves the original workflow exception if cleanup also fails.

Metal GPU trace capture

  • Added /metal on macOS when Defold uses the Metal graphics adapter.
  • Added one-frame and multi-frame .gputrace capture.
  • Added pending, capturing, complete, cancelled, and failed states.
  • Added status polling and early cancellation.
  • Metal capture requires the engine to launch with METAL_CAPTURE_ENABLED=1.
  • Added game.metal_capture with typed status and errors.
  • The Python helper creates the output directory and waits for completion by default.
  • Added documentation for analyzing captures with Xcode and gpudebug, including interactive, scripted, and persistent-session workflows.

Health, lifecycle, and capability negotiation

  • Expanded health information with:
    • native version
    • supported API range
    • per-capability versions
    • engine instance identity
    • process ID where available
    • wall and monotonic start timestamps
    • hashed project and build-configuration identities
    • graphics, HID, window, and headless backend state
  • Added ordered lifecycle stages, including engine registration, health availability, and initial-scene readiness.
  • Capabilities are now derived from the available runtime backends instead of advertised unconditionally.
  • Unsupported routes return explicit 501 unsupported_capability diagnostics.
  • Health and lifecycle remain available when graphics, window, screenshot, scene, or HID backends are unavailable.
  • Added Python API-version validation and name>=version capability declarations.
  • Added IncompatibleApiVersionError and UnsupportedCapabilityError.
  • Added supports(...), require(...), and required capabilities during bootstrap.
  • Cached engine ports now include engine and project identity.
  • The wrapper rejects a cached port reused by another engine instance while accepting a fresh editor registration after a new build.

Editor automation API

  • Added automatic project-editor discovery through .internal/editor.port.
  • Added discovery of registered Defold installations on macOS, Windows, and Linux.
  • Can launch the most recently used valid installation when the project editor is not running.
  • Reuses an existing healthy project editor when possible.
  • Added incremental and clean build-and-run workflows.
  • Added connection to an already-running project engine.
  • Added HTML5 build-and-run command support.
  • Added typed build errors with source positions and ranges.
  • Added editor commands for:
    • fetching libraries
    • hot reload
    • rebundling
    • reloading extensions
    • reloading stylesheets
  • Added debugger start, stop, break, continue, detach, step-into, step-out, and step-over operations.
  • Added structured editor-console snapshots and context-managed streaming.
  • Added editor reference search with environment, language, and query filters.
  • Added scene-resource PNG preview rendering with explicit dimensions or resolution multipliers.
  • Added typed editor preferences with:
    • generated built-in preference constants
    • metadata and descriptions
    • get, set, list, and describe operations
    • support for custom editor-script preference paths
  • Editor operations are validated against the editor’s OpenAPI document before use.
  • On macOS, the wrapper now detects restricted Codex sandboxes before attempting to launch Defold and raises an actionable LaunchError. Healthy existing editors can still be reused.

Diagnostics, logs, gestures, and traces

  • Added a bounded background engine-log collector started automatically by public connection helpers.
  • Added game.logs.tail(...) with substring filtering and retention of the latest 10,000 lines.
  • Retained explicit live log streams and forward-only log reads.
  • Added structured WaitTimeoutError information:
    • last observation
    • elapsed time
    • attempt count
    • scene sequence
    • last explicitly retryable exception
  • Wait helpers now retry only caller-selected exception types instead of hiding arbitrary programming errors.
  • Added interruption-safe cleanup for input, held pointers, log streams, screenshots, profilers, recording, and traces.
  • Added deterministic seeded gesture generation with:
    • bounds constraints
    • duration ranges
    • control-point generation
    • velocity and acceleration limits
    • preflight rejection of impossible gestures
  • Added atomic JSON diagnostic traces containing:
    • initial health, screen, and scene data
    • API actions and receipts
    • generated paths
    • selector failures
    • events, state, and acknowledgements
    • profiler correlation
    • frames and timestamps
    • screenshots and SHA-256 hashes
    • cleanup progress
  • Trace replay is explicitly marked best-effort and records required external prerequisites.

Profiling

  • Consolidated profiling under game.profiler.
  • Added resource profiling helpers.
  • Added bounded multi-frame captures with warmup frames.
  • Added arbitrary-duration background recording.
  • Added scope and counter filtering by name, path, text, thread, or regular expression.
  • Added aggregate timing and value statistics, including percentile data.
  • Added explicit profiler recording finalization and abort hooks.
  • Interrupted Remotery captures now salvage available data and preserve the original interruption.

HTTP API improvements

New or substantially expanded endpoints include:

  • GET /health
  • GET /lifecycle
  • GET/PUT /screen
  • POST /coordinates/convert
  • GET /frame
  • GET /scene
  • GET /nodes
  • GET /node
  • PUT /input/configure
  • POST /input/click
  • POST /input/drag
  • POST /input/drag_path
  • POST /input/key
  • GET /input/status
  • GET /input/pending
  • POST /input/cancel
  • POST /input/flush
  • POST /input/pointer/open
  • POST /input/pointer/move
  • POST /input/pointer/hold
  • POST /input/pointer/up
  • GET /screenshot
  • GET /screenshot/status
  • GET /recording/capabilities
  • GET /recording/status
  • POST /recording/start
  • POST /recording/stop
  • GET /events/cursor
  • GET /events
  • GET /state
  • GET /state/wait
  • POST/GET/DELETE /commands
  • POST /markers
  • POST/GET/DELETE /metal

Additional protocol changes:

  • POST and PUT operations can accept a root application/json object.
  • JSON bodies are bounded by size and nesting depth.
  • Query parameters remain supported and take precedence over duplicate JSON fields.
  • Added structured errors for invalid JSON, oversized bodies, unsupported media types, stale scenes, unavailable backends, input ownership conflicts, recording failures, and capture lifecycle failures.

Fixes

  • Fixed overlapping input requests overwriting active HID state.
  • Fixed held input surviving cancellation, interruption, or lease expiry.
  • Fixed structured mutation helpers bypassing the common request hook.
  • Fixed stale-scene tests and mutations using the superseded query representation.
  • Fixed unsafe collection-root instance identity access during health discovery.
  • Fixed zero scene generations being classified as malformed instead of stale.
  • Fixed cached engine ports connecting to a different restarted process.
  • Fixed cleanup failures masking KeyboardInterrupt and other original exceptions.
  • Fixed screenshot completion relying on file-size polling.
  • Fixed macOS screenshot row inversion.
  • Fixed macOS recordings including window corners.
  • Fixed Windows native recorder callback lifetime and shutdown behavior.
  • Fixed capability reporting on reduced graphics, window, and HID backends.

Documentation and examples

  • Rewrote the native endpoint and Python wrapper documentation for the new APIs.
  • Added examples for:
    • application events, state, commands, acknowledgements, and annotations
    • capability negotiation
    • interruption-safe automation
    • snapshot-safe scene observation
    • coordinate conversion and frame waits
    • structured diagnostic traces
  • Added a native recording platform contract.
  • Added pinned editor OpenAPI and preference-schema fixtures.
  • Expanded unit and live integration coverage for input lifecycle, curved gestures, scene identity, screenshots, capability negotiation, editor workflows, recording, Metal capture, traces, logging, profiling, and cleanup.
Assets 2
Loading