fix(cli,aspire): isolate DB lifecycle and require readiness evidence

[rickylabs]

Summary

Separates one-shot netscript db lifecycle ownership from the resident AppHost and requires endpoint-backed readiness evidence for generated executable resources.

The implementation and acceptance gates are complete. The PR remains draft because the required formal IMPL-EVAL route is credential-blocked; no evaluator verdict is claimed.

Scope

• Archetype / area: Archetype 6 CLI tooling · Aspire lifecycle/readiness
• Closes fix(cli): a read-only netscript db command terminates the resident AppHost #1011
• Closes fix(aspire): executable resources report Healthy without an endpoint readiness check #1012

Slices

• S0 Contract-first research and plan — 5601b8fa0
• S1 Distinct DB-operation AppHost; resident PID/backchannel survives
• S2 Endpoint readiness, zero-report evidence, and live dead-port verdict
• S3 Formal IMPL-EVAL and ready-for-review transition (credential-blocked)

Definition of Done

• Detached DB operations never start, restart, or stop aspire/apphost.mts.
• DB operations use a distinct AppHost identity and invocation-owned lifecycle.
• A live integration gate proves netscript db status preserves the resident PID/backchannel.
• Every generated endpoint-bearing executable receives a readiness probe by default.
• Healthy with zero health reports is distinguishable from verified healthy.
• A live integration gate proves an alive process whose port never binds is not healthy.
• Missing aspire binary produces the explicit unavailable outcome, not an uncaught failure.
• Scoped static, package tests, quality, architecture, and runtime gates are green.
• Separate-session IMPL-EVAL returns PASS (canonical route credential-blocked).
• fix(cli): a read-only netscript db command terminates the resident AppHost #1011 and fix(aspire): executable resources report Healthy without an endpoint readiness check #1012 acceptance boxes are checked with linked evidence.

Validation

• PLAN-EVAL — owner-waived on record; blocked route retained in drift.md.
• CLI package suite — 555 tests passed after correcting two stale string assertions; focused correction suite 24/24.
• Scoped check/lint/fmt — zero findings.
• deno task quality:scan — pass.
• deno task arch:check — pass with pre-existing warnings only.
• Hosted scaffold.runtime job 91672078377 — passed=71 failed=0.
  • database.init passed.
  • behavior.db-status-preserves-apphost passed in 21.998s.
  • behavior.endpoint-readiness passed in 845ms.
• IMPL-EVAL provider canary — blocked: credential absent, non-retryable auth diagnostic, zero evaluator events.

Acceptance evidence

• fix(cli): a read-only netscript db command terminates the resident AppHost #1011: fix(cli): a read-only netscript db command terminates the resident AppHost #1011 (comment)
• fix(aspire): executable resources report Healthy without an endpoint readiness check #1012: fix(aspire): executable resources report Healthy without an endpoint readiness check #1012 (comment)
• Runtime correction: fix(cli,aspire): isolate DB lifecycle and require readiness evidence #1088 (comment)

Harness

• Run dir: .llm/runs/fix-1011-aspire-lifecycle--codex/
• drift.md records both the owner-authorized PLAN-EVAL waiver and the distinct blocked IMPL-EVAL launch.

[rickylabs]

[PHASE: RESEARCH]

Re-baselined #1011/#1012 against main @ ab0fa13fe, merged #1027/#1033, and read #1076 first.

Findings

1. fix(cli): preserve resident AppHost during db commands #1027 removed the explicit resident stop, but detached DB commands still call aspire start on the resident apphost.mts identity; the live PID/backchannel criterion remains unproved.
2. Aspire --isolated permits parallel instances, while describe/stop select by AppHost path. The safe ownership contract therefore needs both a distinct generated DB-operation path and isolated startup.
3. fix(aspire): probe executable HTTP readiness #1033 added probes for apps/services/plugins, but endpoint-bearing tauri/task entries still omit them; zero-report Healthy remains indistinguishable; no dead-port live fixture exists.
4. fix(plugins): order-independent plugin wiring, fail-fast stream producers, and a doctor that can fail #1076's AppHostInspector is the existing observation seam and already handles a missing Aspire binary explicitly.

Evidence

• Run research: .llm/runs/fix-1011-aspire-lifecycle--codex/research.md
• Contract/design commit: 5601b8fa0

Next

• Separate open-model PLAN-EVAL; product implementation remains blocked until PASS.

[rickylabs]

[PHASE: PLAN]

Lifecycle ownership and readiness contracts are locked at 5601b8fa0.

Locked decisions

1. Detached DB commands use a distinct generated aspire/db-operation-apphost.mts identity with --isolated; they never start or stop resident aspire/apphost.mts.
2. The live runtime gate records the resident PID/backchannel, runs netscript db status, and proves the same resident remains observable afterward.
3. Every generated executable that advertises HTTP receives a probe by default, including endpoint-bearing tauri/task entries.
4. The existing AppHostInspector preserves healthReports; Healthy with zero reports becomes warning/unverified, not success or failure.
5. scaffold.runtime gains a live process that never binds its endpoint and asserts a non-healthy report-backed snapshot.

Gates

• Focused DB/generator/inspector/doctor/E2E tests with red proofs.
• Scoped check/lint/fmt wrappers; package tests; quality:scan; arch:check; JSR/doc gates.
• One contention-free scaffold.runtime run at merge readiness.

Next

• Formal PLAN-EVAL in a separate Claude Code + OpenRouter Qwen session.

[rickylabs]

[PHASE: PLAN-EVAL] [VERDICT: BLOCKED]

The required independent formal evaluator did not produce a verdict.

• Route requested: local Claude Code + OpenRouter, qwen/qwen3.7-max, high effort.
• Observed output: Not logged in · Please run /login from the isolated evaluator profile.
• Artifact check: .llm/runs/fix-1011-aspire-lifecycle--codex/plan-eval.md is absent.
• Product implementation: not started; all implementation and validation gates remain NOT_RUN.
• Recorded evidence commit: e5f38a76c.

Next action: restore authentication for the canonical evaluator profile, or obtain an explicit owner
authorization for a documented fallback/waiver. No PLAN-EVAL verdict is inferred from the failed
launch.

[rickylabs]

[PHASE: IMPLEMENT] [SLICE: S1 — #1011 DB AppHost lifecycle]

Commit: 9f0239b2d

Contract landed:

• Detached netscript db operations now use generated aspire/db-operation-apphost.mts, never the
  resident aspire/apphost.mts identity.
• One-shot start is --isolated; lifecycle lock, ownership probe, polling, logs, and scoped stop all
  target only the DB-operation path.
• Interactive db studio remains attached to the resident entry.
• scaffold.runtime now contains a live gate that records resident appHostPid, cliPid, and
  dashboard identity, runs real db status, compares identity, and reopens the original describe
  backchannel.

Red proof against pre-fix behavior:

DbOperationRunner: actual --apphost C:\repo\sample-app/aspire/apphost.mts
expected: C:\repo\sample-app/aspire/db-operation-apphost.mts
HelpersGeneratorPipeline: should include db-operation-apphost.mts ... FAILED (entry absent)

Gate evidence:

deno task --cwd packages/cli test
ok | 554 passed (479 steps) | 0 failed

scoped check: 766 files, 0 findings
scoped lint: 766 files, 0 findings
scoped fmt: 766 files, 0 findings
deno task quality:gate: PASS (quality scan clean; existing doctrine/catalog warnings only)

Opposite-family review: native Claude Opus fallback session
d8cc7066-8900-4c1f-a0bd-2bfe264f48b1 found a symlink false-red in the live gate. Both path sides
now use Deno.realPath with resolve fallback; rereview verdict: REVIEW_PASS.

The owner-authorized Plan-Gate waiver is recorded in the run artifacts. The live
scaffold.runtime gate remains deliberately NOT_RUN until the single merge-readiness pass, so no
#1011 acceptance box is ticked yet.

[rickylabs]

S2 implementation — endpoint readiness

Commit: 9e58b4c92 (fix(aspire): require endpoint readiness evidence)

Contract implemented:

• every generated endpoint-bearing executable (app, and port-bearing tauri/task) receives its configured HTTP readiness probe after endpoint registration;
• AppHostInspector preserves Aspire healthReports instead of collapsing them into healthStatus;
• plugin doctor reports warning for Running / Healthy with zero reports, so absence of readiness evidence is distinguishable from a passing probe;
• scaffold.runtime now generates a live readiness-dead-port task whose process stays alive and never binds its advertised port, then requires non-Healthy status plus at least one health report.

Red proof against pre-fix behavior:

• command: deno test --allow-all packages/cli/src/kernel/templates/aspire/helpers/tests/generators-background-app_test.ts packages/cli/src/kernel/adapters/aspire/apphost-doctor-inspector_test.ts packages/cli/src/public/features/plugins/doctor/doctor-plugin-command_test.ts
• observed: 3 failed for the intended reasons: missing Tauri/task probes, discarded healthReports, and doctor returning healthy instead of warning for an empty report set.
• the missing-Aspire-binary test passed in the same run, proving the Deno.Command/NotFound path remains covered.

Post-fix evidence:

• same focused command: 19 passed (38 steps), 0 failed.
• deno task --cwd packages/cli test: package suite completed successfully, including the full CLI E2E unit suite.
• deno task check: passed.
• scoped check/lint/fmt wrappers over packages/cli (ts,tsx): passed with zero findings.
• deno task quality:scan: ok: true, zero findings.
• deno task arch:check: passed; output contains only pre-existing catalog/doctrine warnings.

No #1012 acceptance box is ticked yet. The once-only live scaffold.runtime proof is deliberately reserved for merge readiness, after the required leak/ownership probe.

[rickylabs]

Merge-readiness discovery/correction

Commit: 1dbb5478b (fix(cli): isolate database Aspire project identity)

The first scaffold.runtime attempt produced a real counterexample to the planned path-only model:

• gate: database.init
• captured artifact: aspire start failed with exit code 7: The --apphost option specified a project that does not exist.
• verified file existed at aspire/db-operation-apphost.mts; Aspire rejected it because the enclosing aspire.config.json identified only apphost.mts.

Correction:

• DB operations now own a complete nested Aspire project at aspire/db-operation/ (its own aspire.config.json, apphost.mts, SDK restore location, tsconfig, and generated helper graph), not a second loose file under the resident project identity.
• detached Aspire does not forward the caller environment to the AppHost, established by a direct start whose AppHost reported Missing NetScript DB operation arguments; the serialized lifecycle lock owner now writes a short-lived local operation-request file, consumed before composition and removed in finally.
• the nested helper graph resolves workspace paths two levels up, verified after an intermediate live FailedToStart showed the wrong aspire/database/postgres workdir.
• live db status then reached Prisma at the generated database and returned the expected exit 1 because the fixture has no migrations. Its actual logs name the database, schema, endpoint, and No migration found; leak-check afterward showed no survivors.

The same live aspire describe also corrected a readiness assumption: Aspire 13.4 emits healthReports as a keyed object. Inspector/doctor and the dead-port E2E gate now preserve/count keyed objects (and tolerate arrays), rather than silently treating real reports as empty.

Validation after correction:

• focused lifecycle/generator/inspector/doctor tests: 20 passed (30 steps), 0 failed.
• full packages/cli package suite: passed.
• scoped check/lint/fmt over 768 CLI TS/TSX files: zero findings.

No acceptance boxes ticked yet; the full runtime retry still must reach both live regression gates.

[rickylabs]

Runtime credential correction

Commit: e8b44955f (fix(cli): preserve database credentials across lifecycles)

The runtime retry passed DB init/generate/seed but the later resident runtime.wait.postgres timed out at 120s. The operation AppHost had been started with --isolated, which explicitly randomizes user secrets while all lifecycles share the configured persistent Postgres data path. The first database lifecycle initializes the persisted cluster with one credential; later isolated lifecycles advertise another.

The DB-operation project already supplies the distinct lifecycle/backchannel identity required by #1011, so --isolated is neither needed nor correct here. It is removed, preserving stable Aspire project secrets across one-shot operations. Focused runner tests pass (12 steps), and the scoped type-check is clean.

The runtime retry still did not reach the #1011/#1012 behavior gates, so no acceptance box is ticked.

[rickylabs]

Implementation correction 9cb9c631d pushed.

Root cause (observed, not inferred): the DB-operation AppHost and resident AppHost are distinct Aspire project identities but attach to the same persistent .data/postgres bind mount. Aspire generated different implicit Postgres passwords. After DB initialization, the resident resource reported 28P01: password authentication failed for user "postgres".

Contract: AppHosts sharing persistent database storage must share one project-scoped credential source. Generated Postgres/MySQL resources now use an explicit secret parameter backed by .data/aspire-secrets/<resource>.password (0700 directory, 0600 exclusive-create file), resolved from the common project root.

Red proof: pre-correction scaffold.runtime passed database.init, then runtime.wait.postgres failed after 120s; aspire describe reported resident Postgres Running / Unhealthy, health report 28P01.

Post-fix evidence from a fresh one-pass run:

• database.init: PASSED 66395ms (migration 20260803104056_init created and applied)
• database.generate: PASSED 94259ms
• database.seed: PASSED 99104ms
• resident runtime.wait.postgres: PASSED 31400ms / resource healthy
• behavior.endpoint-readiness: PASSED; readiness-dead-port observed Running / Unhealthy / 1 reports
• generator suites: 30 steps passed, 0 failed
• scoped fmt/check/lint: 22 files, 0 findings

The same run later failed at the pre-existing users-service health probe: Prisma received an ADO.NET-style DATABASE_URL (Host=...) and reported a raw-query failure. The improved pretty reporter surfaced this diagnostic inline. This is downstream of the CI database.init regression and is being investigated separately before merge readiness.

[rickylabs]

Record correction (orchestrator, post-merge). The Slices / Definition of Done / Validation / Acceptance-evidence checklists in this PR's body are stale. They were written in the S0 planning commit and never refreshed as the slices landed, so the body still reads PLAN-EVAL — pending; implementation hard stop remains in force and Focused tests and scoped wrapper gates — not run yet. That is not what shipped.

Verified against main at merge commit a8a129feb (35 files, +1208/−49):

Deliverable	State on main
packages/cli/e2e/src/application/gates/scaffold/verify-db-status-preserves-apphost.ts	present, 109 lines
packages/cli/e2e/src/application/gates/scaffold/verify-endpoint-readiness.ts	present, 44 lines
Suite registration	BEHAVIOR_DB_STATUS_PRESERVES_APPHOST, BEHAVIOR_ENDPOINT_READINESS in cli-surface.ts
DB lifecycle isolation	kernel/adapters/database/operation-runner.ts +78, new _aspire-compat.ts.template
Readiness inspection	adapters/aspire/apphost-doctor-inspector.ts +10

Gate evidence at merge: close-gate SUCCESS — it fails while any - [ ] remains on a closed issue, and #1011/#1012 were fully ticked with evidence comments. scaffold-runtime (aspire + docker + postgres) SUCCESS, which also confirms the earlier database.init FAILED 486ms regression was fixed rather than skipped. Zero new deno-lint-ignore / as unknown as in non-run-artifact code.

Process gap this exposes, worth fixing beyond this PR: close-gate validates issue checkboxes, not PR-body checkboxes. A PR body can therefore assert an implementation hard stop while the gate goes green and the change merges — leaving the permanent record of a merged PR contradicting what actually shipped. The audit trail is the merged diff and the close-gate result; the body should not be trusted independently of them. Filed as a 0.0.5 follow-up rather than expanding 0.0.4.