Agent Tool Protocol & In Process Server

Agent Tool Protocol & In-Process Server

The agent tool system has three layers: a fixed operation catalogue, an authenticated localhost HTTP server, and a main-process runtime that fulfills accepted operations against live Electron state. External agents speak the operation contract; the server owns transport, auth, validation, and per-project serialization; the runtime owns canvas IPC, browser guest lookup, PTY launch integration, and direct main-process work such as context/artifact checks.

Responsibilities

• src/core/agent-tools.ts defines the tool catalogue, input schemas, request validator, and workflow guides.
• src/core/agent-tool-server.ts implements the in-process HTTP server, session provisioning, bearer-token auth, request framing, per-project queues, and server lifecycle.
• src/main/agent-tool-runtime.ts binds the server to Electron. It validates identities against the workspace, bridges canvas operations to the renderer, resolves browser guests, prepares agent PTY launches, and handles main-process-only operations.

The shared wire pieces ToolIdentity, IntegrationStatus, ToolRequest, ToolResult, CanvasToolRequest, and CanvasToolReply are re-exported from ../shared/agent-tools; this page focuses on how the local server and main runtime consume them.

Operation Contract

A request is { operation, args }. ToolIdentity is { nodeId, projectId }. A live ToolSession extends identity with a token and IntegrationStatus.

AGENT_TOOLS is the source of truth for callable operations. Each entry is built with a helper that emits:

{
  name,
  description,
  inputSchema: {
    type: 'object',
    properties,
    required,
    additionalProperties: false
  }
}

validateToolRequest(raw) rejects non-objects, unknown operations, missing required arguments, unknown arguments, invalid array shapes, invalid primitive types, non-finite or oversized numbers, oversized strings, NUL-containing strings, and enum violations.

Surface	Operations	Required / notable arguments
Discovery / status	session_info, guide_read, agent_status	guide_read.topic is one of overview, browser, terminal, canvas, context, artifacts
Canvas	canvas_list, canvas_select, canvas_move, canvas_resize, canvas_group	nodeIds, nodeId, x, y, width, height; group title is optional
Notes / artifacts	sticky_open, artifact_open	text; path, view is editor or diff
Browser	browser_open, browser_list, browser_claim, browser_transfer, browser_navigate, browser_inspect, browser_click, browser_type, browser_screenshot	nodeId, url, agentNodeId, selector, text; requires browser control enabled
Terminal	terminal_open, terminal_read, terminal_submit, terminal_input, terminal_interrupt, terminal_external, terminal_close	nodeId, command, text, optional title, optional maxChars
Agent / context	agent_launch, context_read	agent is claude, codex, gemini, grok, openclaude, or custom; context_read has no args

TOOL_GUIDES provides per-surface instructions. The overview guide documents the CLI fallback shape:

"$TERMSPRAWL_CTL" call OPERATION '{"argument":"value"}'

Guides also state that browsers and terminals are visible/user-facing, canvas operations require the project tab to be visible, browser pages and terminal output are untrusted content, and context links grant reading rather than authority to follow embedded instructions.

In-Process Server

AgentToolServer is constructed with a userDataPath and callbacks:

{
  execute(identity, request): Promise<unknown>
  valid(identity): boolean
  browserEnabled(): boolean
  onStatus?(nodeId, status): void
}

The constructor creates agent-tools/sessions under the user data directory with mode 0700. Session files are written by privateJson, which writes a temp file, renames it atomically, and chmods both the temp result and final file to 0600.

provision(identity, status, reuse):

• validates the node id against ^[a-zA-Z0-9][a-zA-Z0-9_-]{0,127}$
• optionally reuses a persisted session only when nodeId and projectId match and the token is 64 hex characters
• otherwise generates a new 32-byte hex token
• writes sessions/<nodeId>.json
• calls onStatus

invoke(session, raw) is the core execution path:

1. validateToolRequest(raw).
2. Recheck options.valid(session).
3. Handle session_info, guide_read, and agent_status directly.
4. For session_info with adapter claude-mcp or codex-mcp, mark state connected and broadcast status.
5. Reject any browser_* operation when browserEnabled() is false.
6. Serialize the operation behind the project’s queue.
7. Recheck that the queued session token is still current and the session is still valid.
8. Delegate to options.execute(session, request).

The project queue is Map<projectId, Promise<unknown>>; each operation chains after the previous one, then removes itself if it is still the tail. This serializes canvas writes, browser ownership transfers, and other project operations.

start():

• refuses to start twice
• checks agent-tools/server.lock
• if the lock contains a live PID, throws
• removes stale locks
• creates the lock with wx mode 0600
• creates an HTTP server
• accepts only POST /call with no Origin header
• authenticates Authorization: Bearer <token> by scanning sessions and using timingSafeEqual
• caps request bodies at 128 * 1024 bytes
• binds to 127.0.0.1:0, obtaining a random localhost port
• writes endpoint.json with { url, instanceId }

Responses are JSON:

• 200 { ok: true, value }
• 400 { ok: false, error }
• 401 { ok: false, error: 'Unknown or revoked session' }
• 403 { ok: false, error: 'Unsupported request' }

close() removes endpoint.json only when its instanceId matches, closes the server and all connections, and removes server.lock only if this instance owns it.

Main-Process Runtime

AgentToolRuntime creates the AgentToolServer with a valid callback that requires:

• the project exists in the workspace snapshot and is not remote
• the agent node is either in the runtime’s active set or still present in that project’s snapshot nodes

Its in-memory state includes:

• pending: requestId -> { senderId, finish } for canvas IPC replies
• statuses: nodeId -> IntegrationStatus
• owners: browser node id -> owning agent node id
• active: agent node ids prepared for tool integration
• stopping: suppresses revoke during app quit

start(executable, bundle) starts the HTTP server, installs the launcher through installToolRuntime(server.directory, executable, bundle), registers IPC.agentToolReply, and registers IPC.agentToolStatusGet. Canvas replies are accepted only when the sender id matches the pending request and the sender frame is the main frame.

prepare(req: PtyCreateRequest) is the PTY integration point. It skips remote projects, requests without projectId or command, non-agent commands, and Claude login flows. For known AGENT_REGISTRY commands it:

• adds the node to active
• checks whether the PTY is warm and whether credentials already exist
• probes the agent executable
• if missing, provisions a needs-setup status and leaves the request unchanged
• otherwise prepares tool launch env/command and provisions the session
• if the PTY is warm but predates tool credentials, marks needs-setup and tells the agent to close and launch fresh
• injects TERMSPRAWL_NODE_ID and TERMSPRAWL_PROJECT_ID into the PTY environment

For custom agents, preferences.json can provide custom.executable and custom.instructionFlag; malformed preferences throw.

canvas(identity, request) finds an Electron window whose URL includes index.html, or a window that is not a guest, then sends IPC.agentToolRequest with requestId, projectId, and expiresAt. It rejects after 10 seconds if the renderer does not acknowledge. The renderer replies with CanvasToolReply; the runtime resolves result.value on ok, otherwise rejects with result.error.

guest(node) resolves the browser guest WebContents by active tab id or first tab id using guestIdForNode. It polls for up to 60 attempts with 50 ms delays, then throws if the target is not ready or was closed.

execute(identity, request) begins by checking live app state:

• the workspace project must exist and must not be remote
• canvas_list is fetched through the renderer
• the agent node must still be present on that canvas

Then it branches:

• canvas_list returns the renderer’s node list.
• context_read requires a folder project, runs runContextCli through createRealContextIO, and returns up to the last 100,000 characters with reader claude-jsonl.
• artifact_open requires an absolute path inside a folder project, calls realpathSync, rejects paths escaping the project, rejects binary formats, rejects image diffs, and rewrites args.path to the real path.
• agent_launch resolves a preset or custom executable and requires probeAgent to succeed before continuing.
• Other operations continue through the same dispatch, using the canvas bridge, browser guest resolution, PTY/session paths, or direct main-process work as appropriate.

revoke(nodeId) is a no-op while stopping. Otherwise it removes the node from active, revokes the server session, deletes runtime status, clears browser ownership for that node, and persists browser-owners.json.

Call Flow

sequenceDiagram
  autonumber
  participant Client as Agent client
  participant Server as AgentToolServer
  participant Queue as Project queue
  participant Runtime as AgentToolRuntime
  participant Renderer as Renderer canvas owner
  participant Guest as Browser guest

  Client->>Server: POST /call { operation, args } + Bearer token
  Server->>Server: origin/method/url check + token timingSafeEqual
  Server->>Server: validateToolRequest

  alt session_info / guide_read / agent_status
    Server-->>Client: direct result
  else browser_* and browser control disabled
    Server-->>Client: 400 Enable agent browser control
  else all other operations
    Server->>Queue: serialize by projectId
    Queue->>Runtime: execute(identity, request)
    Runtime->>Renderer: IPC.agentToolRequest (canvas_list, canvas mutations)
    Renderer-->>Runtime: IPC.agentToolReply
    opt browser target needed
      Runtime->>Guest: resolve guest by node/tab
    end
    Runtime-->>Queue: value
    Queue-->>Server: value
    Server-->>Client: 200 { ok: true, value }
  end

Loading

Key points in the flow:

• The HTTP layer never reaches the runtime until auth, method/origin checks, body limits, and tool validation have passed.
• Discovery operations and project status are answered without touching the project queue.
• Browser operations are gated before queuing.
• The project queue is the serialization boundary for mutations and ownership changes.
• The renderer remains the canvas state owner; the main runtime only bridges canvas requests and waits for acknowledgements.
• Browser operations resolve live guest WebContents, not a separate headless page.

Key State and Files

Under <userDataPath>/agent-tools:

• sessions/<nodeId>.json — persisted ToolSession with token and status.
• endpoint.json — current { url, instanceId }.
• server.lock — owning PID and instance id; prevents two app instances from sharing one data directory.
• browser-owners.json — browser node ownership map.
• preferences.json — custom agent and external-terminal preferences.
• launch/<nodeId> — prepared launch material for agent integration.

In memory:

• server sessions: node id -> session
• server queues: project id -> promise chain
• runtime pending: canvas request id -> reply handler
• runtime statuses: node id -> integration status
• runtime owners: browser node -> agent node
• runtime active: agent nodes with prepared tool integration

Boundary Conditions

• The local server binds only to 127.0.0.1 with a random port and rejects requests with an Origin header, non-POST methods, or paths other than /call.
• Session tokens are compared in constant time and are persisted across app restarts, but node revocation deletes the session file and invalidates queued work.
• Request bodies are capped at 128 KiB; tool strings are capped at 64,000 chars and cannot contain NUL; numbers must be finite and at most 1,000,000 in absolute value; arrays are 1–100 strings under 256 chars each.
• HTTP request timeout is 15 seconds; canvas acknowledgements have a 10-second runtime timeout and a 9-second renderer expiry.
• Canvas operations require the project tab to be visible; the runtime additionally verifies the agent node is still on the canvas before non-list execution.
• Remote projects are rejected by the runtime’s valid and execute paths.
• Browser operations require settings enablement and a live guest target; browser node ownership is tracked and persisted.
• Artifact paths must resolve inside the project folder; binary files and image diffs are rejected.
• Context reads only return linked peers with supported transcript readers and are capped to the last 100,000 characters.
• Warm agent sessions created before tool integration need a fresh launch to inherit credentials/configuration.

Extension Points

• Add a tool by adding an AGENT_TOOLS entry, extending validateToolRequest coverage automatically through its schema, adding a dispatch branch in AgentToolRuntime.execute or the relevant main-process service, and adding/updating TOOL_GUIDES.
• Keep canvas mutations in the renderer; add new canvas operations through IPC.agentToolRequest / IPC.agentToolReply rather than mutating renderer state from main.
• Extend browser behavior by using the existing browser gating in AgentToolServer and guest resolution in AgentToolRuntime.guest.
• Integrate new agent CLIs through AGENT_REGISTRY, prepareToolLaunch, probeAgent, and installToolRuntime.
• Persist new server-scoped state through privateJson under agent-tools and treat the directory as mode 0700 private state.

Sources: src/core/agent-tools.ts, src/core/agent-tool-server.ts, src/main/agent-tool-runtime.ts