✨ Replace free-port probing with attached services

.github/workflows/ci.yml

@@ -105,6 +105,13 @@ jobs:
             --component-dir packages/core/components \
             --raw
 
+      - name: Smoke test attached-service ping-pong with the compiled binary
+        run: |
+          ./dist/xmd test smoke-test/attached-service-ping-pong.test.md \
+            --component-dir smoke-test \
+            --component-dir packages/core/components \
+            --raw
+
       # The script installs a second copy of core beside a repository component.
       # The declaration must cross into the bundled engine so the failure prints
       # and execution continues.

README.md

@@ -168,22 +168,30 @@ Built-in modifiers:
 - `silent` - execute but suppress rendered output.
 - `persist` - keep resources created by an eval block alive for the component lifetime.
 - `timeout=30s` - cancel a long-running block.
-- `daemon` - start a long-running subprocess tied to the component scope.
+- `daemon` - start an arbitrary fixed-configuration subprocess tied to the component scope.
+- `service=name` - start an attached service and publish its live loopback endpoint.
+- `ephemeral` - reconstruct live eval state without writing a journal event.
 
 LLM sampling is not a fence modifier — it happens through the `<Sample>` component installed by provider middleware (see [Provider components](#provider-components)).
 
 ## Eval blocks
 
-`eval` blocks run in a shared VM context and binding environment for the current component.
+Plain `eval` blocks run in a shared durable binding environment for the current component.
 
 ````md
-```ts eval
-const port = yield* findFreePort();
-const baseUrl = `http://127.0.0.1:${port}`;
+```bash service=server exec
+node handshake-compatible-server.js
 ```
 
-```bash daemon exec
-./server --port {port}
+```ts persist ephemeral eval
+import { callService } from "./client.ts";
+
+const endpoint = server;
+yield* Sample.around({
+  *sample([request]) {
+    return yield* callService(endpoint, request);
+  },
+});
 ```
 ````
 
@@ -193,18 +201,19 @@ Highlights:
 - Bare `{name}` interpolation inside executable block content reads from eval bindings.
 - `output("...")` lets an eval block render text into the document.
 - `renderChildren()` and `render(markdown)` let eval blocks render nested content intentionally.
+- `ephemeral eval` reruns during live execution and partial replay, exports only invocation-local live bindings, and cannot render output.
+- Attached-service endpoints are available only to `ephemeral eval`; they never enter interpolation, durable effect descriptions, or the journal.
 
 ## Provider components
 
 The repo includes reusable markdown components (in `packages/core/components/`) that demonstrate the provider pattern:
 
 - `AnthropicProvider.md`
 - `OllamaProvider.md`
-- `LlamafileProvider.md`
 - `Sample.md`
 - `Instruction.md`
 
-These components combine `eval`, `daemon`, readiness checks, and `Sample` middleware so a document can talk to a cloud or local model server without custom runtime wiring.
+These components combine eval and `Sample` middleware so a document can talk to a cloud or already-running local model server without custom runtime wiring. A local process provider attaches a handshake-compatible command through `service=<binding>` and authenticates it with the XMD service handshake protocol.
 
 [`packages/core/examples/hello-world.md`](packages/core/examples/hello-world.md) shows the pattern combining a cloud model (Claude) and a local model (Ollama). Provider docs currently need the built-in components on the search path:
 

architecture.md

@@ -43,6 +43,8 @@ Existing documents and code get aligned to this section retroactively.
 | suspension | a durable wait: a crash restarts into the same wait |
 | Workspace | the provider-neutral, run-owned environment that supplies retained filesystem, repository, process and working-directory capabilities to a workflow |
 | ephemeral | a replay classification for an operation, context or attachment that runs again to reconstruct live execution; its result is not substituted from the journal and it owns no durable workflow state |
+| live binding | an execution-owned value reconstructed ephemerally for the current document execution; it is visible only to constructs that explicitly consume the live binding overlay and never enters interpolation or the journal |
+| attached service | a scoped host process that publishes its authenticated loopback endpoint through the XMD service handshake protocol and remains supervised for the lifetime of its service attachment |
 | effect transaction | the single atomic SQLite transaction that publishes one Workspace-local mutation together with that effect's journal result |
 | external effect | an effect whose provider-owned outcome cannot participate in the Workspace SQLite transaction and therefore requires a stable identity and provider reconciliation |
 | checkpoint | a completed journal boundary associated with the logical Workspace root visible after that effect |
@@ -583,6 +585,29 @@ execution. No middleware sees it; it is never the document's own outcome.
   journal is parsed, never trusted — an unreadable record is refused, not
   coerced.
 
+## Replay-safe live services
+
+An attached service belongs to the document execution that attaches it. The
+host owns port selection, process spawning, XMD service handshake
+authentication, supervision and teardown. Shared runtime code reaches that
+behavior through the provider-neutral `API.Service`; it never imports a host
+process or networking API.
+
+The service publishes a frozen loopback endpoint as a live binding. Live
+bindings form an overlay on the component's durable eval bindings: `ephemeral
+eval` reads both and may add live bindings atomically, while ordinary `eval`,
+code-block interpolation and journal serialization read durable bindings only.
+A live binding cannot shadow a durable binding, and a durable value cannot be
+replaced by a live one.
+
+Service attachment and `ephemeral eval` execute again during partial replay so
+the current process and middleware chain are reconstructed. A completed
+document replay returns its recorded result without expanding the document and
+therefore starts no service. Workflow execution installs a non-delegating
+`API.Service` denial provider: a workflow cannot reach an inherited host
+adapter, because a run-owned durable service requires stable identity and
+reconciliation rather than an execution-owned live process.
+
 ## State ownership
 
 All state is scoped to the operation that owns it, so it is torn down when
@@ -637,6 +662,10 @@ Status is measured against main.
 | `Git.revParse()` | verifies and resolves one Git revision expression contextually | built on main |
 | workflow run storage | creates or compatibly finds one run by public run ID, and retains its identity, state, document executions and filtered journal | built on main |
 | caller-owned storage transaction | publishes several changes, including journal events, in one transaction nothing else enlists in | built on main |
+| `API.Service` / `startService()` | creates an authenticated, supervised loopback service attachment through a provider-neutral operation | built on main |
+| `service=<binding>` | publishes the attachment's endpoint into the live binding overlay for its invocation | built on main |
+| `ephemeral eval` | reconstructs live middleware and bindings without a journal entry | built on main |
+| workflow service denial | prevents workflow documents from inheriting an ordinary host service adapter | built on main |
 | `xmd workflow start` / `xmd workflow resume` | starts or resumes a workflow run from the CLI | defined in `specs/workflow-workspace-spec.md`, unbuilt; the lookup it resumes through is built |
 | implicit workflow Workspace | retains provider-neutral filesystem, repository and attachment state by run ID | defined in `specs/workflow-workspace-spec.md`, unbuilt (#218) |
 | Repository / Worktree / transactional Git effects | compose named checkouts and publish local mutations with their journal result | defined in `specs/workflow-workspace-spec.md`, unbuilt |

packages/cli/src/bun-service.ts

@@ -0,0 +1,17 @@
+import { randomBytes } from "node:crypto";
+import process from "node:process";
+import type { Operation } from "effection";
+import { inheritedEnvironment, installHostService } from "./service-host.ts";
+
+export function useBunService(): Operation<void> {
+  return installHostService({
+    token: () => randomBytes(32).toString("hex"),
+    environment: () => inheritedEnvironment(process.env),
+    stdout(bytes) {
+      process.stdout.write(bytes);
+    },
+    stderr(bytes) {
+      process.stderr.write(bytes);
+    },
+  });
+}

packages/cli/src/bun.ts

@@ -11,6 +11,7 @@ import process from "node:process";
 import { API } from "@executablemd/runtime";
 import { compileDataUri } from "@executablemd/core";
 import { runXmd } from "./cli.ts";
+import { useBunService } from "./bun-service.ts";
 
 const ENTRYPOINT = fileURLToPath(import.meta.url);
 
@@ -29,5 +30,5 @@ await main(function* (args) {
     },
     { at: "min" },
   );
-  yield* runXmd(args);
+  yield* runXmd(args, useBunService);
 });

packages/cli/src/cli.ts

@@ -407,6 +407,8 @@ interface DocumentMode {
   props?: Record<string, Json>;
 }
 
+export type HostServiceInstaller = () => Operation<void>;
+
 /**
  * Run one document and report how it finished.
  *
@@ -415,7 +417,11 @@ interface DocumentMode {
  * what the process status is. Rendered output, the --verbose journal echo, and
  * a value root's JSON line are the document's own output and stay.
  */
-function* runDocument(config: DocumentConfig, mode: DocumentMode): Operation<Result<void>> {
+function* runDocument(
+  config: DocumentConfig,
+  mode: DocumentMode,
+  installService: HostServiceInstaller,
+): Operation<Result<void>> {
   const { root, componentDir, verbose, journal, raw, secretDetection } = config;
 
   // Every CLI invocation starts from an empty stream. --journal writes
@@ -497,6 +503,10 @@ function* runDocument(config: DocumentConfig, mode: DocumentMode): Operation<Res
   // alone. Reading the mode costs no document effects.
   const valueRoot = !mode.testing && (yield* readsValue(root));
 
+  // Native service authority belongs only to document execution. Help,
+  // document inspection, and the agent worker never enter this scope.
+  yield* installService();
+
   const execution = yield* execute({
     ...root,
     stream,
@@ -557,9 +567,13 @@ function* runDocument(config: DocumentConfig, mode: DocumentMode): Operation<Res
  * only be caught out here. That is what lets a directory run continue past a
  * document whose resources failed to release.
  */
-function* runScopedDocument(config: DocumentConfig, mode: DocumentMode): Operation<Result<void>> {
+function* runScopedDocument(
+  config: DocumentConfig,
+  mode: DocumentMode,
+  installService: HostServiceInstaller,
+): Operation<Result<void>> {
   try {
-    return yield* scoped(() => runDocument(config, mode));
+    return yield* scoped(() => runDocument(config, mode, installService));
   } catch (error) {
     return Err(error instanceof Error ? error : new Error(String(error)));
   }
@@ -592,7 +606,11 @@ interface TestConfig extends Omit<DocumentConfig, "root"> {
  * end. A single document behaves exactly as it always has: one reported
  * failure, no heading, no summary.
  */
-function* test(config: TestConfig, args: string[]): Operation<void> {
+function* test(
+  config: TestConfig,
+  args: string[],
+  installService: HostServiceInstaller,
+): Operation<void> {
   const patterns = readPatternFlags(args);
   if (patterns.missingValue) {
     console.error(
@@ -621,7 +639,11 @@ function* test(config: TestConfig, args: string[]): Operation<void> {
       return;
     }
     announceSecretDetection(config.secretDetection);
-    const result = yield* runScopedDocument({ ...config, root: { path } }, { testing: true });
+    const result = yield* runScopedDocument(
+      { ...config, root: { path } },
+      { testing: true },
+      installService,
+    );
     if (!result.ok) {
       reportFailure(result.error);
       yield* exit(1);
@@ -660,6 +682,7 @@ function* test(config: TestConfig, args: string[]): Operation<void> {
         componentDir: componentSearchPath(document, target.root, config.componentDir),
       },
       { testing: true },
+      installService,
     );
     if (!result.ok) {
       reportFailure(result.error, document.relativePath);
@@ -935,7 +958,7 @@ function* resolveRunProps(
  * `process.stdout` and `node:fs/promises`. Routing those through contextual
  * APIs is #156.
  */
-export function* runXmd(args: string[]): Operation<void> {
+export function* runXmd(args: string[], installService: HostServiceInstaller): Operation<void> {
   // First, so that no later scanner — help, properties, agent flags — can
   // mistake the inline document's own text for an option.
   const evalFlags = readEvalFlags(args);
@@ -1025,6 +1048,7 @@ export function* runXmd(args: string[]): Operation<void> {
             denyAll: config.denyAll,
           },
         },
+        installService,
       );
       if (!result.ok) {
         reportFailure(result.error);
@@ -1049,7 +1073,7 @@ export function* runXmd(args: string[]): Operation<void> {
         yield* exit(1);
         break;
       }
-      yield* test(command.config, evalFlags.rest);
+      yield* test(command.config, evalFlags.rest, installService);
       break;
     }
     case "test-agent":

packages/cli/src/compiled-service.ts

@@ -0,0 +1,17 @@
+import { randomBytes } from "node:crypto";
+import process from "node:process";
+import type { Operation } from "effection";
+import { inheritedEnvironment, installHostService } from "./service-host.ts";
+
+export function useCompiledService(): Operation<void> {
+  return installHostService({
+    token: () => randomBytes(32).toString("hex"),
+    environment: () => inheritedEnvironment(process.env),
+    stdout(bytes) {
+      process.stdout.write(bytes);
+    },
+    stderr(bytes) {
+      process.stderr.write(bytes);
+    },
+  });
+}

packages/cli/src/compiled.ts

@@ -10,6 +10,7 @@ import process from "node:process";
 import { API } from "@executablemd/runtime";
 import { compileDataUri } from "@executablemd/core";
 import { runXmd } from "./cli.ts";
+import { useCompiledService } from "./compiled-service.ts";
 
 await main(function* (args) {
   // The base providers for this host. `at: "min"` puts them beneath ordinary
@@ -26,5 +27,5 @@ await main(function* (args) {
     },
     { at: "min" },
   );
-  yield* runXmd(args);
+  yield* runXmd(args, useCompiledService);
 });

packages/cli/src/deno-service.ts

@@ -0,0 +1,17 @@
+import { randomBytes } from "node:crypto";
+import process from "node:process";
+import type { Operation } from "effection";
+import { inheritedEnvironment, installHostService } from "./service-host.ts";
+
+export function useDenoService(): Operation<void> {
+  return installHostService({
+    token: () => randomBytes(32).toString("hex"),
+    environment: () => inheritedEnvironment(process.env),
+    stdout(bytes) {
+      process.stdout.write(bytes);
+    },
+    stderr(bytes) {
+      process.stderr.write(bytes);
+    },
+  });
+}

packages/cli/src/deno.ts

@@ -13,6 +13,7 @@ import process from "node:process";
 import { API } from "@executablemd/runtime";
 import { compileDataUri } from "@executablemd/core";
 import { runXmd } from "./cli.ts";
+import { useDenoService } from "./deno-service.ts";
 
 const ENTRYPOINT = fileURLToPath(import.meta.url);
 
@@ -31,5 +32,5 @@ await main(function* (args) {
     },
     { at: "min" },
   );
-  yield* runXmd(args);
+  yield* runXmd(args, useDenoService);
 });

packages/cli/src/node-service.ts

@@ -0,0 +1,17 @@
+import { randomBytes } from "node:crypto";
+import process from "node:process";
+import type { Operation } from "effection";
+import { inheritedEnvironment, installHostService } from "./service-host.ts";
+
+export function useNodeService(): Operation<void> {
+  return installHostService({
+    token: () => randomBytes(32).toString("hex"),
+    environment: () => inheritedEnvironment(process.env),
+    stdout(bytes) {
+      process.stdout.write(bytes);
+    },
+    stderr(bytes) {
+      process.stderr.write(bytes);
+    },
+  });
+}

packages/cli/src/node.ts

@@ -17,6 +17,7 @@ import process from "node:process";
 import { API } from "@executablemd/runtime";
 import { compileTempFile } from "@executablemd/core";
 import { runXmd } from "./cli.ts";
+import { useNodeService } from "./node-service.ts";
 
 const ENTRYPOINT = fileURLToPath(import.meta.url);
 
@@ -36,5 +37,5 @@ await main(function* (args) {
     },
     { at: "min" },
   );
-  yield* runXmd(args);
+  yield* runXmd(args, useNodeService);
 });