✨ Replace free-port probing with attached services

[taras]

Why

Free-port probing reserves nothing, so a provider can lose the selected port before its process binds. It also gives durable replay no way to distinguish a live invocation-local endpoint from persisted state.

Closes #381.
Closes #137.

What changes

Before:

Providers selected a port with findFreePort, launched a process separately, and could persist or replay an endpoint that no longer named a live service. The first implementation also delayed unterminated service stdout until teardown, allowed durable eval to collide with a live service binding, and did not translate observable process teardown failures.

After:

A service=<binding> exec block starts a scoped attached service. The child binds loopback port 0, authenticates one exact handshake record with a per-attachment token, and publishes a frozen endpoint only to the current invocation's live environment. ephemeral eval reruns on partial replay and can consume that endpoint without journaling it.

Ordinary stdout is forwarded incrementally, while bounded handshake candidates are suppressed. Durable eval rejects exports that collide with live bindings before live execution, replay restoration, or an eval journal append. Ephemeral eval rejects durable collisions but may update an existing ephemeral live binding. Observable process teardown failures become ServiceTeardownError and compose with active execution failures through invocation teardown aggregation.

Completed replay does not start the attached service, and workflow execution explicitly denies inherited host service providers.

How it works

service exec → scoped host child → prefix-aware handshake observer
             → invocation-local live binding → ephemeral eval consumer
             → supervised exit/cancellation → ordered teardown

The stdout observer retains only bytes that can still match XMD_SERVICE_READY: at line start. A mismatch releases those bytes immediately; only a real handshake candidate is buffered, with a finite bound. The durable operation validates live-name collisions before either execution or replay restoration, while consuming a matching replay entry so partial replay remains coherent.

Review guide

Start with: packages/cli/src/service-host.ts and packages/core/tests/ephemeral-service.test.ts

Then review:

1. packages/core/src/live-env.ts and packages/core/src/eval-handler.ts for namespace collision rules
2. packages/durable-streams/effect.ts for validation before live execution or replay restoration
3. packages/cli/tests/service-host.test.ts for byte-level observation, concurrency, cancellation, and teardown aggregation
4. packages/cli/tests/service-document.test.ts for real-process supervision and nested lifetimes
5. specs/executable-mdx-spec.md for the durable/live contract and test matrix

Look carefully at:

• immediate ordinary stdout forwarding without handshake-token leakage
• replay consumption without restoring a colliding durable export or appending another eval event
• preservation of an active execution failure when service teardown also fails
• real-process cleanup after cancellation, projected failure, and unexpected exit

What must stay true

• Live attached-service endpoints never enter durable events — enforced by the separate live environment and checked by core and CLI journal tests.
• Attached-service publication refuses durable and live names; durable eval refuses live names; ephemeral eval refuses durable names while allowing live updates — enforced before the operation that would create each collision.
• Partial replay restarts service attachment, while completed replay starts nothing — checked with real child processes.
• Ordinary stdout remains live and byte-exact; valid, invalid, split, duplicate, and oversized handshake records remain suppressed — checked at the byte observer and real-process boundaries.
• Service attachments belong to the invocation scope — checked through success, execution failure, unexpected exit, cancellation, and nested-provider teardown.
• Workflow code cannot inherit a host process provider — enforced by a non-delegating minimum handler.

How to verify it

• ./dist/xmd test smoke-test/attached-service-ping-pong.test.md --component-dir smoke-test --component-dir packages/core/components --raw runs the real two-attachment ping-pong document through the compiled binary and production host adapter.
• deno test --allow-all packages/cli/tests/service-document.test.ts proves journal isolation, partial/completed replay, supervision, cleanup, and nested lifetime behavior with real processes.
• deno test --allow-all packages/cli/tests/service-host.test.ts proves incremental handshake observation, concurrent attachments, cancellation, and teardown errors.
• deno task verify passes all ten applicable commands concurrently: lint, check, Deno tests, JSR dry-run, Node 22 typecheck/tests, Bun tests, docs, site check, and site build.
• The verifier confirms the tracked tree remains unchanged.
• Focused coverage includes the compiled smoke document and proves concurrent attachments, a two-attached-service ping→pong→ping exchange whose endpoints remain live-only across partial replay, live unterminated stdout, split byte forwarding, split/duplicate/bounded handshake-record suppression, partial-replay collisions, teardown error aggregation, post-handshake cancellation and listener release, unexpected exit during projected content, prompt failure cleanup without restart, and nested teardown ordering.

Scope

Included

• Provider-neutral runtime service API and test stub
• Node, Deno, Bun, and compiled CLI host adapters
• Live invocation environment, service, and ephemeral eval
• Incremental authenticated stdout observation and service teardown translation
• Workflow denial, real attached-service lifecycle coverage, and specification updates
• Removal of findFreePort and the Llamafile-specific provider example

Intentionally unchanged

• daemon remains the fixed-configuration subprocess primitive and does not publish an endpoint.
• The XMD service handshake protocol is process/stdout based and loopback-only.
• Updating an existing ephemeral live binding remains allowed.

New abstractions

• Service, ServiceAttachment, and ServiceEndpoint define provider-neutral scoped attachment.

• The live environment carries invocation-only bindings without entering durable state.

• EphemeralEval gives middleware a scoped, non-journaled eval boundary.

• Durable-operation validation runs before live execution or replay restoration so a rejected operation creates no result event.

• Each new abstraction has multiple concrete uses or a clear justification.

• No speculative functionality is included.

Generated or mechanical changes

• site/routes/docs/exec-eval.tsx was formatted with the site formatter after its documentation update.

Risks and limitations

• Handshake-compatible commands must emit the exact handshake record and continue serving for the owning scope.
• The endpoint is deliberately limited to 127.0.0.1; non-loopback providers are rejected.

Scope confirmation

• Every changed file supports the purpose described above.
• Unrelated cleanup and formatting changes are excluded.
• Generated or mechanical changes are clearly identified.
• The description matches the final diff and test results.

[github-actions]

PR #390: ✨ Replace free-port probing with cooperative services

53 files, +3262 / -1918

Scope

🔴 PR has 5180 lines changed. Split into focused PRs.

🟡 5180 lines changed. PRs under 400 receive more thorough review.

🟡 53 files changed. Are all changes related?

Structural

✅ No structural bloat detected.

Slop

✅ Slop indicators look low.

Static Analysis

✅ Oxlint found no issues.

Correctness

No extraneous code patterns detected.

[taras]

The scope warning is understood. This is one cross-package behavioral migration rather than several independent features:

• 1,487 deleted lines are the retired Llamafile/free-port implementation and its two old suites.
• The replacement tests exercise the same contract at the provider-neutral runtime, real host-process, replay, workflow-denial, and smoke boundaries.
• Runtime API, host adapters, live-binding semantics, and the executable specification must agree in the same landed change; splitting them into separate PRs would leave intermediate branches with either an exported contract that no entrypoint installs or syntax whose replay behavior is undocumented/unverified.

The branch is arranged as three dependency-ordered commits for review: host/runtime API, replay-safe modifiers and integration, then documentation. Every changed file maps to an acceptance criterion in #381, and the aggregate ten-command verifier passes with an unchanged tracked tree.