AI Agents framework: agents as WordPress users, abilities runner, chat with persisted conversations, drag & drop and Send to triggers

[epeicher]

What it does

Ships the AI Agents framework: durable workers that live on the site as real WordPress users, take orders by chat, drag & drop, or the "Send to" right-click menu, and act through the WordPress Abilities API under their own role and capabilities. The remaining trigger intakes (hook, REST endpoint, agent-to-agent) are follow-up work; their kinds are already declared and marked "coming soon" in the UI, so their configuration stores today and the intakes plug in without a storage migration.

Behind the new agents extended option (OS Settings → Features → Extended options, admin-only, default off). With the flag on, a new Agents section appears in My WordPress: create an agent (name, role, description, system prompt), tick the abilities it may use in the Tools pane, configure triggers, and talk to it in the Agent chat window. Every tool call runs as the agent's own user and lands in the standard audit trail (revisions, comments, _edit_lock) attributed to the agent.

Rationale

This is the buildable slice of the direction #240 mocked, with one deliberate storage change: no wp_guideline CPT. An agent is split across exactly two layers:

• Identity: a synthetic wp_users row. Real role, real capability checks, real attribution. Every login path is blocked (authenticate filter, password resets, application passwords), the email is a never-delivered synthetic address, and the wp-admin Users list labels the row "Agent".
• Definition: user meta on that row (_desktop_mode_agent_*): description, instructions, ability allowlist, triggers, model override, rate limit.

Dropping the guideline CPT removes the Gutenberg Guidelines-experiment dependency entirely (no soft-gate, no 412 paths). The accepted trade-off is losing free revisions on prompt edits; instead, every mutation fires desktop_mode_agent_{created,updated,deleted} with before/after values per changed field, so logging plugins get a complete audit trail.

Implementation

Runner (includes/agents/runner.php): desktop_mode_agent_invoke() generates through the same Core AI Client adapter the Copilot uses (desktop_mode_ai_client_generate() over wp_ai_client_prompt()), loops function calls to a hard 8-turn cap, and dispatches each call via WP_Ability::execute() so the ability's own permission_callback and schemas gate it. The whole loop runs with wp_set_current_user() switched to the agent, restored in finally:

$previous_user_id = get_current_user_id();
wp_set_current_user( $user->ID );
try {
    $result = desktop_mode_agent_runner_loop( ... );
} finally {
    wp_set_current_user( $previous_user_id );
}

Tool schemas advertised to the model are projected through the Copilot's desktop_mode_ai_normalize_tool_schema() (providers 400 the whole request over one tool with a top-level oneOf/anyOf/allOf). Per-agent hourly rate limits ride a transient counter (default 60/hour, filterable). A desktop_mode_agent_runner_generate pre-filter short-circuits the AI Client, which is how PHPUnit exercises the full loop network-free.

WordPress 7.0 AI Client: all generation runs on the AI Client introduced in WordPress 7.0 — every agent turn builds a wp_ai_client_prompt( $messages ) fluent chain (using_system_instruction() for the agent's prompt, using_function_declarations() to advertise abilities as tools via the SDK's FunctionDeclaration/FunctionCall/FunctionResponse DTOs, generate_result() to run it). Provider, model, and API keys are delegated entirely to the Core Connectors API (Settings → Connectors plus the official Anthropic/Google/OpenAI provider plugins); Desktop Mode pins none of them. The UI's availability probe is the client's own feature detection (is_supported_for_text_generation()). The JS-side wp-ai-client package is deliberately not used — generation stays server-side inside /invoke, matching Core's guidance for distributed plugins. Two places we go beyond the stock client: the function-calling tool loop, and thought-part handling (provider plugins don't round-trip Gemini/Anthropic thinking signatures, so desktop_mode_ai_strip_thought_parts() plus transcript replay keeps multi-turn tool loops working).

Tools: the picker is a view over wp_get_abilities() with honest read-only vs mutating badges from meta.annotations.readonly. Unlike the Copilot (read-only only), agents may be granted mutating abilities; the compensating controls are the explicit allowlist set by an edit_users human plus the agent's role. Five new abilities ship: desktop-mode/get-post and desktop-mode/get-media (read-only), plus the mutating trio desktop-mode/update-post, desktop-mode/update-media (alt text / title / caption / description), and desktop-mode/create-post (status hard-forced to draft — it can never publish); mutating abilities are reachable only through an agent allowlist.

REST (/desktop-mode/v1/agents): CRUD + /invoke + abilities/trigger-kinds/hooks/roles catalogues. Reads and invokes default to edit_posts, writes to edit_users (all filterable); role assignment is constrained to a whitelist intersected with the acting user's get_editable_roles().

Client: an agent entity kind registered through the existing registerEntityKind() seam in the My WordPress bundle (list, Define/Tools/Triggers panes, create flow, live provider probe against /ai/status), plus a new lazy agent-run-window bundle for the chat window, fed through the cross-bundle desktop-mode/agents-chat shared store.

Triggers: chat, drag & drop, and Send to are wired. A drop is a chat whose message carries the dropped entity: agents accept entity drops on their rows in the Agents section, on their user tiles on the wallpaper (via the files layer's tile-payload-handler seam, gated synchronously by agentDragKinds inlined into the user-file payload from the agent's drag trigger), and in the open chat window. Send-to rides the site folder's tile-context-menu filter seam: agents whose triggers include a send-to row appear as "Send to " entries in the tile context menus (posts, pages, media, users), gated by the trigger's entityKinds; the users grid was refactored onto the same filter seam the posts and media grids already used. The remaining kinds (hook, endpoint, agent) carry wired: false in the trigger-kind catalogue and render as disabled "coming soon" options in the Triggers pane; their intakes land in later phases. All of them collapse to the same desktop_mode_agent_invoke() engine, and desktop_mode_agent_completed is the chaining seam.

Chat window UX: WhatsApp-style avatars (agent on the left, the viewer's own avatar on the right, fed from the window config), agent answers rendered as markdown through a new shared src/markdown.ts (extracted from the AI assistant; supports headings, lists, bold/italic/code, safe links, thematic breaks, and applies inline tokens per line so stray asterisks cannot pair across lines), a readable in-flight bubble instead of a tiny chip, and a past-conversations sidebar ("+ New chat" on top, then the saved conversations with the agent's avatar, derived title, active highlight, and a hover delete with confirm). The Agents window detail head also gains Open profile (opens the agent's user-edit window) and Send to Desktop (creates a wallpaper tile on the first free grid cell), and agent rows in the list are draggable straight onto the desktop, producing the same shortcut tile.

Default agents (includes/agents/defaults.php): a fresh site gets a ready-to-use roster the first time an edit_users admin loads wp-admin with the flag on — tl;dr, Comment Concierge, Localizer (author role, drafts only), SEO Medic, and Alt Text Librarian, each with a full system prompt, ability allowlist, and chat + send-to + drag triggers. Seeding runs once per site and ONLY when no agents exist at all; an install that already built its own roster gets the seeded flag without any rows. Allowlisted abilities that aren't registered (the ai/* family from the AI experiments plugin) are skipped at tool-build time, so they light up automatically when the provider plugin lands.

Call-to-action buttons: agents that need the user's confirmation return renderable buttons instead of asking for a typed reply. The runner constrains every final answer to a { text, call_to_actions } JSON schema through the AI Client's structured output (as_json_response()), and a system-prompt appendix teaches the convention, so existing agents pick it up without prompt edits. Each action carries a label, a style (primary/secondary/danger → wpd-button variants), and a reply — the literal message posted back as a visible user turn when the button is pressed, so the stored history shows exactly what was approved. Parsing is lenient (non-JSON answers pass through verbatim with no actions), sanitization caps 4 actions / 40-char labels / 500-char replies, only the latest message's buttons are live, and spent buttons persist disabled with the conversation.

Persisted conversations (includes/agents/conversations.php): each conversation is one post of the private desktop_mode_chat post type — post_author is the human, messages as JSON in post_content, agent id in meta, title derived from the first user message. The posts table was chosen over user meta deliberately (WordPress loads all of a user's meta into cache on any meta read, so fat transcripts would tax every request). REST CRUD lives under /agents/conversations and is strictly owner-only — foreign rows read as 404 so ids cannot be probed, and not even administrators can read another user's chats through this API. The shared dispatcher auto-saves after every completed exchange (chat, drag, and send-to all persist; failures are swallowed so persistence can never break the conversation). Caps: 100 conversations per user (desktop_mode_agent_conversation_cap filter) with LRU pruning, 200 messages per conversation, tool-call outputs dropped on store. Each /invoke still replays the visible transcript via the history param, now capped at 50 turns (desktop_mode_agent_history_turn_cap filter).

Testing instructions

npm run test:php -- --filter='Tests_DesktopMode_Agents'   # 64 tests
npm run test:js                                            # includes the agents, send-to, chat window, and markdown suites

Manual, on a WP 7.0+ site with an AI connector configured:

1. OS Settings → Features → Extended options → tick Enable AI agents, reload. On a site with no agents, the five default agents appear in the Agents section on the next wp-admin load.
2. My WordPress → Agents → create an agent (role author, prompt like "You audit posts"). Tick desktop-mode/get-post + desktop-mode/update-post in Tools.
3. Press Chat, ask "Fetch post 1 and tighten its title". Expect an answer with an expandable tool-call trace, and the post's revision attributed to the agent user.
4. Verify the blocks: the agent appears in wp-admin Users with the bot avatar and "Agent" type, logging in as it fails, and no application passwords can be created for it.
5. Without a connector, the section shows a warning notice pointing at Settings → Connectors and Chat is disabled; definitions remain editable.

Drag & drop (verified live in the browser against a real connector):

6. In the agent's Triggers pane, add the Drag & drop trigger and tick the entity kinds it should accept (nothing ticked = every kind). Without this trigger every drop shows the no-drop cursor.
7. Drag the agent's user tile from My WordPress → Users onto the wallpaper — the resulting user tile is a live drop target.
8. Drag a media tile from My WordPress → Media onto that desktop tile, onto the agent's row in the Agents section, or into the open Agent chat window. Expect the Send to agent chip while hovering, then the chat window opening with the composed drop message and the run streaming into the transcript (source: 'drag' reaches desktop_mode_agent_completed).
9. Dropping a kind outside the trigger's filter (e.g. a post onto a media-only agent) and dropping the agent's own user tile onto itself are both rejected.

Send to and window buttons (verified live in the browser against a real connector):

10. In the agent's Triggers pane, add the Send to (right-click menu) trigger and tick the entity kinds. Right-click a matching tile in the site folder (posts, pages, media, users) and pick Send to : the chat window opens with the composed message and the run lands in the transcript (source: 'send-to').
11. In the Agents window, Open profile opens the agent's user-edit window and Send to Desktop places the agent's tile on the first free desktop grid cell (an agent already on the desktop gets its existing tile moved there instead of duplicated).
12. The trigger-kind picker renders WordPress hook, REST endpoint, and Agent-to-agent as disabled "coming soon" entries.

Persisted conversations (verified live in the browser against a real connector):

13. Chat with an agent, then press + New chat and chat again: the sidebar lists both conversations, clicking the older one reloads its transcript (and re-targets the chat to its agent), and the list survives a full page reload.
14. Hovering a sidebar row reveals a delete control; deleting the open conversation also clears the transcript. Conversations are owner-only — another user's list is empty and their ids read as 404.

Call-to-action buttons (verified live in the browser against a real connector):

15. Ask an agent to propose a change and request approval (e.g. "Propose a TL;DR for post 860 and ask for my approval before applying it"): the answer arrives with Approve/Cancel buttons instead of a typed-confirmation request. Pressing Approve posts the action's reply as a visible user message, the agent applies the change through its tools, and the spent buttons stay visible but disabled.

[AllTerrainDeveloper]

Checking!