Skip to content

ActionUI demo apps v0.7 Pre-release

Pre-release
Pre-release

Choose a tag to compare

View all tags
@abra-code abra-code released this 24 Jul 08:09
v0.7
d69fbf0

ActionUI stopped being "SwiftUI from JSON with an Android experiment attached" and became a three-renderer system: the same JSON document now drives SwiftUI on Apple, Jetpack Compose on Android, and DOM/CSS in the browser - plus a new add-on architecture that lets third-party element types plug into the engine without living in core.

At a glance

Area Change
ActionUIWeb New. Full browser renderer, 57 elements, mobile/touch adaptation
ActionUIAndroid From partial port to feature parity: elements, providers, add-ons, native chrome
Add-ons New. Public registration API + 5 add-ons + a host test app
Core (Apple) Swift 6, new cross-platform features, new host APIs
Examples New. 4 complete cross-platform example apps
Tooling Verifier add-on schemas, skill packaging, ActionUIViewer aggregator

1. ActionUIWeb - a third renderer

The largest single piece of work in this release. ActionUI JSON now renders in a browser as plain DOM + CSS, with app logic in JavaScript. No build step, no runtime dependencies, plain ES modules. The source layout deliberately mirrors the Swift and Android trees file-for-file (Common/, Helpers/, Views/, one file per element) so the three renderers stay diffable.

Element coverage (57 views). Layout stacks and lazy stacks, ScrollView / ScrollViewReader, Grid / LazyVGrid / LazyHGrid, Form / Section / GroupBox / LabeledContent / DisclosureGroup, the full control set (Slider, Stepper, Picker with segmented/radioGroup/menu styles, SecureField, TextEditor, DatePicker, ColorPicker, ProgressView, Gauge, Toggle), the shape primitives, Label/Image on a shared SF-to-Material glyph resolver, AsyncImage, VideoPlayer, WebView, Canvas, GeometryReader, LoadableView, Link/ShareLink/ContentUnavailableView/EmptyView/Group.

Containers and navigation. List in all three modes (heterogeneous children, homogeneous itemType, data-driven template), Table with the rows API and drag-to-resize columns, TabView/Tab, Menu as a top-layer popover, NavigationStack + NavigationLink, NavigationSplitView with column widths, toolbars with an overflow menu, and the app menu bar as a top app bar.

Runtime surface parity. setElementProperty visual-property bridge, runtime structural mutation (insertElement / insertRow / removeElement, including into TabView / Menu / NavigationStack), programmatic row selection, window-level dialogs and modals, element-level sheet / fullScreenCover / popover, window-level toast, page lifecycle hooks, the animation modifier over armed CSS transitions, searchable, overlay/background decoration subviews, frame(alignment:), file upload via drag & drop, and opt-in browser back/forward through the History API.

Pluggable Map. Dependency-free embed by default, with opt-in providers for MapLibre, Google Maps, and Apple MapKit JS.

Mobile and touch. A substantial adaptation pass: an input-modality layer, sheets/popovers/menus presented as bottom action sheets on a phone, sidebarAdaptable TabView collapsing to a bottom tab bar, responsive NavigationSplitView collapse, left-edge swipe-back, pull-to-refresh, swipe actions, safe-area insets on edge chrome, touch-sized targets and corner radii, textContentType autofill, inset-grouped Forms, and a :web stacked-card mode for Table on narrow screens.

Performance and quality. content-visibility plus incremental row rendering for lazy containers, List and Table; multi-window as multiple in-document surfaces; a debug mode for unmapped symbols; and a development-only Node test suite (node --test, zero deps, 142 tests).

The demo is a NavigationSplitView shell with each section in its own JSON file, loaded through LoadableView. Try it live at https://abracode.com/ActionUIWeb/demo/ - no install, no local server.

2. Android reaches feature parity

Android went from a partial port to a peer of the Apple renderer.

Elements ported: Grid (custom column-aligning layout), LazyVGrid/LazyHGrid, Gauge, AsyncImage (dependency-free), WebView, VideoPlayer (with an audio panel for audio-only sources), Canvas (Skia rendering), GeometryReader, ControlGroup, Map, ScrollViewReader, plus template-based data-driven containers.

Map as split provider modules - :map-osm (Leaflet/OpenStreetMap, the default) and :map-google (maps-compose), with the shared contract in core. This is the pattern the Apple add-on architecture later mirrored.

Modifier and chrome parity: the accessibility modifiers, disabled/buttonStyle/controlSize through an inherited control environment, element-level sheet/fullScreenCover/popover, overlay/background decoration subviews, textSelection, multilineTextAlignment, labelsHidden, onAppear/onDisappear, openURLActionID, contextMenu, transition, searchable, swipe actions, pull-to-refresh, and the animation modifier on a new host-mutable property API.

Native chrome: control-chrome, toolbar and menu icons through the Material Symbols font; a window-level toast as a native bottom Material snackbar; NavigationSplitView per-pane toolbars, sidebar icons and selected-row content tint; a demoApp launcher icon.

Correctness work: SwiftUI-faithful stack layout and frame/Picker parity, layer blending matched to Core Graphics, semantic-color resolution to adaptive theme colors, safe-area handling for un-scaffolded roots, reactive disabled and foregroundStyle/tint, runtime structural mutation, a scroll guard instead of a crash on an unbounded axis, trailing commas in JSON, and compileSdk standardized to 36.

Android also gained its own add-on modules (:addon-cachedimage, :addon-richtext) and an add-on demo app.

3. Add-on architecture

Custom view-type registration became a public, documented API (ActionUIRegistry.shared.register(_:as:) plus the public ActionUIViewConstruction contract). An add-on compiles against ActionUI but does not link it; the host links both and calls the add-on's register() once at launch. Registration is exposed as a plain C entry point so C/C++/Obj-C hosts can use it too. Each add-on ships its own verifier schema, picked up by --schema-dir.

Five add-ons ship in this release:

  • ActionUIQuickLook - the first add-on: an embedded in-process Quick Look pane (QLPreviewView on macOS, QLPreviewController on iOS/visionOS), with drag-selectable text in the macOS preview.
  • ActionUIChat - the Chat element (see below).
  • ActionUIDiff - a Diff element plus a standalone DiffView component (line diff, hunks, gutters).
  • ActionUIRichText - a RichText element rendering a whole Markdown document into one native, selectable, table-aware-copyable text view.
  • ActionUICachedImage - a CachedImage element with off-main fetch/decode/downscale, a two-tier memory + disk cache, and no reflow on hydration.

Supporting infrastructure: ActionUIAddOnTestApp (a multiplatform host with an add-on picker), and ActionUIViewer moved into its own add-on-linking aggregator package (core cannot link add-ons without a circular dependency).

Four components were extracted into standalone sibling repos and are now consumed by version: ChatView, RichText, DiffView, AsyncImageCache.

4. ActionUIChat - a generic chat element

The most feature-dense add-on. One Chat element backs both AI-agent chat and person-to-person chat; the transport and appearance differ, not the view.

  • Transports behind a registry: local (scripted echo), ACP - launches any Agent Client Protocol agent as a subprocess over stdio (validated against OpenCode and claude-code-acp), openai-sse - streams any OpenAI-compatible /v1/chat/completions endpoint, and a local peer-to-peer transport.
  • Streaming Markdown message bodies from a from-scratch engine (no third-party dependency), plus standalone image items.
  • Agentic surfaces: folded reasoning "Thoughts", tool-call cards that mutate through their pending/in-progress/completed/failed lifecycle, a permission gate pinned above the composer, an evolving task plan panel, a token/cost usage line, and model/mode indicators that become menus when the agent offers choices.
  • Agent-proposed file diffs rendered inside tool cards through DiffView.
  • Composer slash-command menu driven by the transport's advertised commands.
  • Session transcript persistence and restore - the transcript is serializable data, persisted per finalized entry and restored by injecting it into states["content"]; readOnly is a pure viewer mode.
  • Dual-alignment (P2P) rendering - two-party transcripts, paging, optimistic-id reconciliation, and connection-state composer gating.
  • Non-visual operational settings (protocol, transport) are host-injected at runtime into states["config"], so documents stay static and secrets never live in JSON. A transport-command security guard backs this.

The chat implementation itself now lives in the standalone ChatView package; ActionUIChat is a thin element-glue wrapper over it.

5. New cross-platform features in core

Each of these landed on Apple, Android, and Web:

  • swipeActions - leading/trailing swipe-to-reveal row actions on List.
  • Pull-to-refresh for List and ScrollView.
  • searchable on List and NavigationStack.
  • Window-level toast (native bottom Material snackbar on Android, styled overlay elsewhere).
  • transition - insert/remove animation, the structural partner of animation. Shorthand string, typed dict (opacity, slide, scale, move, offset, asymmetric), or a combined array.
  • contextMenu / contextMenuPreview subviews (long-press / right-click).
  • ignoresSafeArea / safeAreaInset.
  • Color as a View and SwiftUI's <color>.opacity(<fraction>) syntax.
  • Semantic colors - Apple semantic color names resolve to adaptive theme colors on Android and Web.
  • Programmatic Table/List row selection (by index, by content, and clear).
  • DatePicker displayedComponents (date / hourAndMinute / dateAndTime).
  • Template mode for LazyVGrid / LazyHGrid.

6. Swift 6

The whole package and Xcode project are on Swift 6 language mode with zero warnings. The core target is data-race-safe, deferred view construction uses main-actor bindings, and test-quality warnings are cleared. ActionUIViewer's screenshot capture offers both the legacy no-permission path and a ScreenCaptureKit option, with the deprecation confined behind a protocol witness.

7. Host API and adapter additions

New APIs, exposed consistently through the Swift, Obj-C, C, C++, JSCore, WebKitJS, Node.js, and Python bindings:

  • selectElementRowByIndex / selectElementRowWithContent / clearElementSelection
  • getContentSizeLimits - exposes a window content's SwiftUI size limits to AppKit hosts
  • presentToast / dismissToast

ActionUIMenuBar is now its own SPM-first module, split out of ActionUIAppKitApplication, with per-item targeting, SF Symbol icons, and macOS 26 icon inheritance.

WebView on Apple gained two interchangeable backings selectable by the host: a .native WKWebView wrapper (new default; works on the package baseline and avoids the find-on-page teardown crash) and the existing SwiftUI WebKit.WebView.

8. Example apps

A new Examples/ folder of complete little applications - each loads a JSON skeleton, registers action handlers, and supplies logic in a thin native host, on all three platforms from one shared JSON:

  • TemperatureConverter - the canonical first loop.
  • TipBillSplitter - Slider + Stepper + currency formatting, with a non-SwiftUI Apple shell (AppKit/UIKit hosting ActionUI's SwiftUI tree).
  • UnitConverter
  • BodyMetrics

9. Tooling and documentation

  • Verifier: --schema-dir for add-on element schemas, is_actionui_json.py detection, a Python test suite, and schema updates across the board.
  • Skill: add-on packaging, plus a refresh from real-world usage feedback.
  • Release packaging: The distributed demo bundle now ships ActionUIAddOnTestApp alongside ActionUISwiftTestApp and the ActionUIViewer CLI, plus the ActionUIWeb demo - included in the archive and emitted as a separate upload bundle, so the browser demo can be tried from a hosted URL with no local server (live at https://abracode.com/ActionUIWeb/demo/).
  • Symbol map: a regen-symbol-map.sh pipeline and refreshed sf_to_material.map, sourced from the standalone sf-symbols-material-map repo.
  • Documentation gained a MenuBar JSON guide, per-element schema/template files for the new elements (Color, Toast), and per-add-on documentation bundles that mirror core's layout.

Breaking changes

  • foregroundColor is removed. Use foregroundStyle. foregroundColor was the retired SwiftUI name; Apple and Android only ever read foregroundStyle, and web honored foregroundColor on the static path only - so 34 JSON files were colored on web and silently uncolored everywhere else. Every platform now standardizes on foregroundStyle.
  • Android compileSdk is 36. Hosts building against an older SDK need to bump.

Notable fixes

  • Material-symbol font memory blowup (one shared font buffer across axis instances).
  • Toolbar chrome now respects runtime hidden, reactively.
  • Always-black toast on Apple platforms.
  • Programmatic navigationPath push for standard NavigationStack content.
  • Row double-click on Table and List.
  • Multi-digit template column substitution on Apple.
  • background / cornerRadius / clipShape now apply after frame (fill, not hug).
  • Android setElementState type guard rejecting same-logical-type collections.

Platform support

Platform Minimum
macOS 14.6+
iOS / iPadOS 17.6+
watchOS 10.6+
tvOS 17.6+
visionOS 2.6+
Android 12.0+
Web Any modern browser (ES modules, no build step)

Known gaps

  • ActionUIChat: advanced agentic surfaces (terminals, multi-session) are not implemented.
  • ActionUIWeb: a few perf/exotic items remain deferred (see ActionUIWeb/README.md).
  • ActionUIChat has not been ported to Android or Web (porting plans exist in Documentation/).
Assets 3
Loading
0 Join discussion