Skip to content

AI-native desktop: the idea ledger #1

Description

@etdebruin

Reef already has the bones — ⌘K generation, Fix-with-AI, the planned reef.ai bridge and intents (DESIGN.md §7–8). This issue collects the ideas that push it from "a desktop with an AI feature" to "a desktop that assumes a model is always present." Each item is scoped to be pick-up-able on its own; check it off or split it into its own issue when work starts.

Software as utterance (the disposable end)

  • Ephemeral apps. The single-file static HTML default makes generation cheap enough that apps don't need to be kept. "Compare these two CSVs" spawns a throwaway app that dissolves when its window closes (or offers to stay). Reframes the desktop: apps aren't installed, they're said. Needs: a --ephemeral generation path that writes to a temp dir instead of userData/apps/, and a keep/dissolve prompt on window close.
  • Data-first generation. Linked folders already exist; extend to linked data. Drop a CSV, SQLite file, or JSON API URL on the canvas and ⌘K generates the UI around it. The prompt becomes "here's my data, give it a face." Needs: drop-target on the canvas, data sample injected into the generation prompt, REEF_DATA_DIR wiring for the generated app.
  • The prompt is part of the source. Store the generating conversation as prompt.md in every generated app folder. Makes apps forkable seeds: "make me one like notes, but for recipes" is a remix of a known-good prompt plus a diff, not a cold start. Cheap — agent.js already has the conversation in hand when it writes the folder.

Apps that keep growing (the durable end)

  • Point-and-mutate editing. Edit Live (M5) is planned as chat-beside-app; go one further and make the running window itself the prompt surface. Right-click any element in a frame → "change this" → the agent gets the element's DOM path plus the app source. The gateway's folder-watch reload closes the loop in under a second. Needs: a context-menu hook in the SDK preload that captures the element path.
  • Self-healing apps. The supervisor captures real stderr and the crash panel has Fix-with-AI. Make the loop autonomous for generated (never linked) apps: crash → agent reads log → patches → hot reload → only surface the crash panel if the fix fails twice. createFixer's never-delete rule and the linked-app confirmation click already draw the safety line.
  • Vision-checked generation. npm run shot proved the model looking at its own output pays for itself (invisible icons, dead WebSocket). Fold it into ⌘K: after writing the app, screenshot the rendered frame and let the model iterate on visual defects before the icon materializes.

AI as a system service (reef.ai, expanded)

  • A typed AI surface, not just complete(). reef.ai.extract(text, schema), reef.ai.embed(text), and — now that src/core/policy.js handles mic — reef.ai.transcribe(stream). Apps generated by the agent can use the agent, keyless, capability-gated per manifest. Generated apps become smarter than their author's one-shot because intelligence is a utility like storage.
  • Desktop-wide semantic search. The planned shared SQLite store plus an embeddings table = AI-native Spotlight: "that thing I wrote about tide pools" searches across every app that opted its data in. Compounds the intents story — another thing separate localhost apps structurally can't offer.
  • Natural-language automation. "Every morning, pull yesterday's mileage into my journal" — a scheduler where the action is a prompt executed against apps through intents. The intents registry gives the model a verb list; the scheduler gives it a clock. Depends on intents (v1.5).

The desktop as an agent target (the inversion)

  • Reef as an MCP server. Expose generate_app, list_apps, open_app, invoke_intent over MCP so external agents (Claude Code included) can drive the desktop. Note the symmetry: an app's declared intents (add:link, play:track) are already tool definitions — every app the user installs silently extends what any connected agent can do. The desktop becomes legible in both directions: AI writes the apps, and the apps are callable by AI. Largely the same work as the intents system already scheduled for v1.5.

Suggested sequencing

The typed reef.ai surface and prompt.md provenance are cheap and compound everything else. Ephemeral apps are the demo. Intents-as-MCP-tools is the moat, and it overlaps almost entirely with the v1.5 intents work.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions