Skip to content

alpha.11 fix-train: plan + research (PLAN-EVAL gate)#155

Merged
rickylabs merged 4 commits into
mainfrom
harness/alpha11-fixtrain-plan
Jun 27, 2026
Merged

alpha.11 fix-train: plan + research (PLAN-EVAL gate)#155
rickylabs merged 4 commits into
mainfrom
harness/alpha11-fixtrain-plan

Conversation

@rickylabs

Copy link
Copy Markdown
Owner

alpha.11 fix-train — plan + research (PLAN-EVAL gate)

Plan artifacts only — no framework source. This PR carries the harness PLAN-EVAL deliverable for the alpha.11 fix-train (#141), addressing the Storefront Ch.1 eye-test blockers (#153, published @netscript/cli@0.0.1-alpha.10). Implementation follows on PLAN-EVAL PASS, via WSL Codex daemon-attached slices.

Artifacts: .llm/tmp/run/alpha11-fixtrain--plan/research.md + plan.md.

Slice topology

Slice Lane Scope
A CLI-core Codex F-3 --version from package · F-4 wire dormant DryRunFileSystemAdapter (write-free dry-run)
B type-soundness Codex F-15a/b (prefer template rewrite to existing QueryClientPort methods over widening SDK) · verify F-15c is publish-only drift
C interactive + cache Codex new public surface--cache (default on) + --cache-backend redis|garnet|deno-kv (default redis) · wire CliffyPromptinit (first consumer) · broad interactive prompts for all missing options · per-backend scaffold emission · tutorial→redis
E health + e2e Codex F-14 plain GET /health via ServiceBuilder.withHealth() · add :3001 service probe to scaffold.runtime (also the F-13 diagnostic)
D doc-truth docs aspire runaspire start · F-6 callout correction · F-8 https · F-12 · minors
F install pin docs F-1 dynamic alpha pin + banner

Scout-grounded reversals (why this plan differs from the raw eye-test backlog)

  • F-6 is a doc fix, not a code reorder. db init/generate/seed self-provision Aspire (DbOperationRunner.executeDetached()aspire start detached). The CLI next-steps order is intentionally correct; the tutorial Ch.2 callout is the wrong artifact.
  • F-13 is diagnosed by Slice E's new probe. The runtime e2e never HTTP-probes the scaffolded service on :3001 (only plugin APIs 8091-8094) — the blind spot that let F-13 stay green. Green-on-Linux under aspire start ⇒ F-13 is Windows-aspire run-specific ⇒ fixed by Slice D.
  • Interactive init is dormant infra, not a new build. PromptPort + CliffyPrompt exist, wired into zero commands; init becomes the first consumer.

User decisions folded in (2026-06-27)

  • Install stays on alpha (dynamic pin, not cut 0.0.1).
  • Cache is CLI-managed, default redis.
  • Interactive init is a regression to fixbroad scope (prompt for all missing options).
  • Docs aligned from the e2e as canonical, not just aspire runstart.

Next

OpenHands minimax-M3 PLAN-EVAL (separate session) is the hard gate. No Codex implementation slice begins before PASS.

Refs #141 #153 #138

🤖 Generated with Claude Code

…ockers)

PLAN-EVAL deliverable for the alpha.11 fix-train (#141) addressing the
Storefront Ch.1 eye-test blockers (#153). Six slices: A CLI-core (F-3/F-4),
B scaffold type-soundness (F-15), C interactive-init + cache feature
(F-11, new public surface), D doc-truth alignment driven by the e2e
(F-6/F-8/F-12), E service health + e2e :3001 coverage (F-14/F-13),
F install pin + alpha banner (F-1).

Grounded by two code-truth scouts: F-6 is a doc fix not a code reorder
(db commands self-provision Aspire via DbOperationRunner); F-13 is
diagnosed by a new e2e service-health probe; interactive init reuses the
dormant PromptPort/CliffyPrompt adapter (init = first consumer).

Plan artifacts only; no framework source. Implementation follows on
PLAN-EVAL PASS via WSL Codex daemon-attached slices.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014gW4zfhMMQU6txC828ijct
@rickylabs

Copy link
Copy Markdown
Owner Author

@openhands-agent model=openrouter/minimax/minimax-m3 output=pr-comment

use harness

You are the PLAN-EVAL evaluator (harness Plan-Gate, separate session from the generator). This is a hard gate: no implementation slice may begin until you return PASS. Evaluate the alpha.11 fix-train plan on this PR branch (harness/alpha11-fixtrain-plan, off main). The PR carries plan artifacts only — no framework source.

SKILL

Activate and follow these repo skills before evaluating:

  • .agents/skills/netscript-harness — harness orchestration, the 8-phase model, evaluator separation, and the PLAN-EVAL protocol. This is your governing skill.
  • .llm/harness/evaluator/plan-protocol.md — the PLAN-EVAL instructions you must execute.
  • .llm/harness/gates/plan-gate.md — the Plan-Gate checklist you enforce.
  • .llm/harness/evaluator/verdict-definitions.mdPASS vs FAIL_PLAN definitions.
  • .agents/skills/netscript-doctrine — package/plugin archetype, public-surface, and gate doctrine (Slice C adds a new public CLI surface: --cache / --cache-backend; Slice E touches the service health surface).
  • .agents/skills/netscript-cli — CLI/scaffold command surface (init flags, interactive prompts, scaffold emission) for sanity-checking Slices A/C/E.
  • .agents/skills/netscript-deno-toolchain + .agents/skills/jsr-audit — for the type-soundness (Slice B) and publish-surface reasoning.
  • aspire skill — for the aspire start vs aspire run correctness claims in Slices D/E and the F-6 self-provisioning disposition.

What to read

  1. .llm/tmp/run/alpha11-fixtrain--plan/research.md — code-surface map, e2e↔tutorial command drift, the two code-truth scout resolutions, slice topology.
  2. .llm/tmp/run/alpha11-fixtrain--plan/plan.md — the 6 slices (A CLI-core, B type-soundness, C interactive+cache, D doc-truth, E health+e2e, F install-pin), gate set, debt, and the locked open items.

What to evaluate

Run the full Plan-Gate checklist. In particular, adjudicate:

  1. Slice decomposition — are A/B/C/D/E/F correctly scoped, independently committable/gateable, and dependency-ordered? Is the Claude-supervises / Codex-implements / docs-lane split correct (Slice C/E framework source = Codex; D/F = docs lane)?
  2. F-6 disposition — research claims db init/generate/seed self-provision Aspire via DbOperationRunner.executeDetached(), so F-6 is a doc fix, not a code reorder. Verify against the cited code (packages/cli/src/kernel/adapters/database/operation-runner.ts) and confirm or reject.
  3. F-13 diagnosis — plan treats the :3001 service-not-served defect as Windows-aspire run-specific and adds a scaffold.runtime :3001 health probe (Slice E) as the diagnostic, fixed via Slice D's aspire runaspire start. Is that a sound plan given the e2e passes green on Linux?
  4. New public surface (Slice C)--cache (default on) + --cache-backend redis|garnet|deno-kv (default redis), interactive prompts for all missing options via the dormant PromptPort/CliffyPrompt, per-backend scaffold emission. Is the public-surface change doctrine-conformant and adequately specified (defaults preserved for CI non-interactive)?
  5. Type-soundness (Slice B) — is "rewrite the template to existing QueryClientPort methods" preferred correctly over widening the SDK, and is the F-15c "publish-only drift, verify-not-fix" disposition acceptable or a gap?
  6. Gate set + debt — are the listed gates (deno task check/lint/scoped fmt, scaffold.runtime e2e incl. the new :3001 probe, e2e:cli) sufficient to certify each slice, and is the deferred-item/debt list complete?

Output

Write your verdict to .llm/tmp/run/alpha11-fixtrain--plan/plan-eval.md (commit it to this PR branch) and post the summary as a PR comment. Emit exactly one of:

  • PASS — plan is implementation-ready; list any non-blocking advisories.
  • FAIL_PLAN — list each blocking deficiency with the specific slice/section and the concrete correction required.

Do not commit framework source, deno.lock, or unrelated churn. Preserve lock hygiene. Two FAIL_PLAN cycles maximum before escalation.

@rickylabs

rickylabs commented Jun 27, 2026

Copy link
Copy Markdown
Owner Author

OpenHands Agent — Completed

Model: openrouter/minimax/minimax-m3
Provider: OPENROUTER
Job status: success
Trace: .llm/tmp/run/openhands/pr-155/run-28284573816-1 (committed to the PR branch)
Artifact: openhands-agent-28284573816-1

Summary

PLAN-EVAL verdict: PASS

Evaluated the alpha.11 fix-train plan against the Plan-Gate checklist, with targeted code-truth verification against current main. The plan is implementation-ready. Six non-blocking advisories follow.

Changes

  • Added .llm/tmp/run/alpha11-fixtrain--plan/plan-eval.md (98 lines, committed to harness/alpha11-fixtrain-plan as 0676eea5).
  • Verified all four code-truth claims in research.md against current main:
    • F-6: DbOperationRunner.executeDetached() does self-provision Aspire (verified packages/cli/src/infra/database/operation-runner.ts:76-99,128-131,195-217). Doc fix, not code reorder — confirmed.
    • F-13: init-orchestrator.ts:111 emits aspire run for local-import mode; e2e uses aspire start. Windows-only defect, fixed via Slice D. Sound.
    • F-14: defineService() already calls builder.withHealth() unconditionally (packages/service/src/presets/define-service.ts:205), and /health is in DEFAULT_ANONYMOUS_PREFIXES. The research claim "scaffolded service is oRPC-only and never calls it" is stale for current main — recorded as advisory S0: initial public repo genesis #1 (downgrades Slice E's template work from required to verify-only; new :3001 e2e probe remains the primary deliverable).
    • F-15: getQueryState exists on @tanstack/query-core 5.101.0; template-rewrite to existing QueryClientPort methods is the preferred path. F-15c publish-only drift disposition is acceptable; self-resolves on alpha.11 republish.
  • Adjudicated all six items the task asked me to rule on (slice decomposition, F-6 disposition, F-13 diagnosis, Slice C new public surface, Slice B type-soundness, gate set + debt) — all resolved as PASS with non-blocking advisories.
  • No framework source, deno.lock, or unrelated churn committed. Lock hygiene preserved.

Validation

  • Re-read .llm/harness/evaluator/plan-protocol.md, .llm/harness/gates/plan-gate.md, .llm/harness/evaluator/verdict-definitions.md in full.
  • Re-read research.md (114 lines) and plan.md (214 lines) in full.
  • Code-truth grep verification against current main:
    • packages/cli/src/infra/database/operation-runner.tsexecuteDetached() + Aspire provisioning
    • packages/cli/src/kernel/adapters/init/init-orchestrator.ts:111aspire run for local-import
    • packages/service/src/presets/define-service.ts:205builder.withHealth() wiring
    • packages/service/src/builder/service-builder-impl.ts:354-363createHealthHandler
    • packages/service/src/auth/auth-middleware.ts:23DEFAULT_ANONYMOUS_PREFIXES
    • packages/sdk/src/ports/query-client.ts:35-63QueryClientPort surface
    • packages/cli/src/public/features/init/init-command.ts:39,57--yes and --ci flags
    • packages/cli/src/kernel/assets/service/main.ts.template:10defineService usage
    • packages/cli/src/public/features/root/public-command-tree.ts:102 — hardcoded version: '1.0.0' (F-3)

Responses to review comments or issue comments

The PR review comments were empty at the time of evaluation (pr-review-comments.json: []); no thread replies to author. The PLAN-EVAL verdict itself is posted as a PR comment by the workflow, sourced from the verdict file and this summary.

Non-blocking advisories (carry into the implementer's brief)

  1. F-14 is likely a no-op on current main. defineService().withHealth() is already wired; the research.md claim is stale. Slice E's template rewrite should be a verify-only pass; the new :3001 e2e probe is the primary deliverable. (See plan-eval.md §"Non-blocking advisories S0: initial public repo genesis #1".)
  2. Slice B default path is template-rewrite (not SDK getQueryState widening).
  3. Slice C — check existing CACHE_URL / CACHE_* env conventions before adding a new key.
  4. Slice F — verify the docs homepage version-injection mechanism first; if it exists, the work is "use the existing mechanism across all tutorials + add alpha banner." If not, a new Lume _data source is needed.
  5. Eye-test Aspire serialization — capture the Windows-cross-OS-:18891 collision in worklog.md for the eye-test follow-up run (cross-linked from fix(release): enable --unstable-raw-imports for whole-workspace publish (alpha.5 republish) #138).
  6. F-15c evidence — record publish-only drift evidence (clean local deno check on packages/fresh + alpha.10 vs local Plugin type comparison) in drift.md when Slice B closes.

Remaining risks

  • F-13 conditional escalation. If the new :3001 e2e probe goes RED on Linux/aspire-start, F-13 is NOT a Windows-only defect and a new service-runtime scaffold slice is needed. Slice E's "if probe RED, escalate" gate is the correct safety valve.
  • Garnet/deno-kv integration thinness. Plan records these as debt; the implementer should ensure the default redis path is fully working for alpha.11 even if garnet/deno-kv constants ship behind follow-up.
  • Plan-vs-actual drift. The four "open for PLAN-EVAL to rule on" items in plan.md §"Open for PLAN-EVAL" are all resolved in plan-eval.md; no re-plan needed. The implementer should track any new drift in .llm/tmp/run/alpha11-fixtrain--plan/drift.md per harness protocol.

Run: https://github.com/rickylabs/netscript/actions/runs/28284573816

- Slice decomposition, dependency order, lane split, gate set, debt list all sound.
- F-6 disposition (db self-provisions Aspire via DbOperationRunner.executeDetached)
  confirmed; doc fix only, not code reorder.
- F-14 advisory: defineService().withHealth() is already wired (current main);
  Slice E's template work is verify-only — only the new :3001 e2e probe is required.
- F-15a/b: template-rewrite preferred over SDK widening (getQueryState exists
  on the underlying @tanstack/query-core 5.101.0).
- F-15c: publish-only drift disposition is correct; self-resolves on republish.
- Slice C public surface (--cache/--cache-backend) is contract-first and
  doctrine-conformant; --ci/--yes escape hatches preserve CI non-interactive.
- Gate set: scoped check/lint/fmt + deno task test + deno doc --lint on changed
  public exports + full e2e:cli scaffold.runtime with new :3001 probe.

Co-authored-by: openhands <openhands@all-hands.dev>
…ict=PASS)

Co-authored-by: openhands <openhands@all-hands.dev>
@rickylabs
rickylabs marked this pull request as ready for review June 27, 2026 22:01
@rickylabs
rickylabs merged commit 6abd73a into main Jun 27, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants