Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions .pi/prompts/compare-specs.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,8 @@ Use these repository-relative homes and no substitutes:

A mission is ordinary-language Markdown for the top-level session playing the simulated user. It is not controller YAML, a Brunch seed, comparison-harness selection, or harness configuration. Never use `.fixtures/seeds/` as mission input. Never put the mission text, file, or path in a harness context or cwd. A harness receives only its approved minimal framing and the natural opening and later answers that the top-level session chooses from mission truth.

Every harness gets a fresh system-temporary external target root outside the controller checkout, scratch run tree, and retained run tree. Before launch, resolve the target and controller paths and verify that they are separate and that its ancestor directories contain no controller-private run material. Brunch still launches from the Brunch repository root and receives the external target through `--workspace`; generic harnesses launch with the external target as their cwd. This placement closes ordinary target-relative ancestor traversal, including the known `../../private-mission.md` path. It is not an OS sandbox and does not claim isolation from unrestricted absolute-path or whole-host discovery.

A saved mission is editable. An approved run is historical evidence: never overwrite an existing run directory, private mission snapshot, `harness-setup.md` snapshot, transcript, target output, or report. Later mission revisions affect future runs only. Historical setup snapshots keep their existing names and bytes; never rename them. Keep retained paths repository-relative; do not retain workstation-absolute paths.

All operator choices and approvals must work through ordinary typed text. If a structured question tool happens to exist, it may improve presentation, but correctness and progress must never depend on it.
Expand Down Expand Up @@ -57,7 +59,7 @@ Run setup is intentionally small, run-specific, and never written into the reusa
- **Brunch:** use built-in Specify mode, plus only the output instruction needed to identify the requested review-ready document and path.
- **Generic harnesses:** use a small instruction to conduct a question-led specification conversation and author the requested review-ready Markdown document. Do not preload mission facts or prescribe conclusions.
3. Preflight only each selected harness's filesystem and adapter prerequisites, including the pinned `pi-interactive-shell` package and cleanup capability where applicable. Do not launch a synthetic conversation to test provider/model access. Report provider/model failure honestly if the real harness launch fails. Check Pi's adapter only when Pi is selected.
4. Allocate a fresh isolated target cwd/session for each selected harness. This is the harness's comparison target, not necessarily the controller process cwd: Brunch's controller must launch from the Brunch repository root and address the fresh target through `--workspace <fresh-target-cwd>`, while a generic harness's structured spawn uses the fresh target cwd as its process cwd. The top-level session remains the shared simulated-user actor, so disclose harness order and shared actor context; do not claim fresh-per-harness actor isolation or equivalence to a rigorous campaign.
4. Allocate a separate fresh system-temporary external target root for each selected harness using the host's standard temporary-directory facility. Resolve its canonical path and fail setup unless it is outside the controller checkout, scratch run tree, and retained run tree and its ancestor directories contain no `private-mission.md`, `harness-setup.md`, or other controller-private run material. This is the harness's comparison target, not necessarily the controller process cwd: Brunch's controller must launch from the Brunch repository root and address the external target through `--workspace <fresh-target-cwd>`, while a generic harness's structured spawn uses the external target as its process cwd. The top-level session remains the shared simulated-user actor, so disclose harness order and shared actor context; do not claim fresh-per-harness actor isolation or equivalence to a rigorous campaign.
5. Display together: the complete private mission; selected harnesses and order; exact per-harness target-visible framing; mission id; collision-safe run id; requested target document path; scratch and retained paths; and each adapter. Clearly label the mission **top-level-session-only** and the framing **harness-visible**.
6. Ask through ordinary text for explicit **approve**, **revise**, or **reject**. Revise and redisplay the complete setup, or reject and stop; do not launch partially.

Expand Down Expand Up @@ -91,8 +93,8 @@ Run selected harnesses sequentially. For each harness:

Generic harnesses author the requested Markdown file themselves in their cwd. An unavailable or mismatched adapter makes the harness failed; never substitute, silently drop it, or reconstruct output.
6. Retain the exact harness-visible initial framing and transcript, including every user answer and decision. This is the disclosure record: any mission fact visible to the harness must have arrived through a natural opening or subsequent answer.
7. Acquire the harness-authored document if it exists. Never author, reconstruct, complete, rewrite, or improve it. Missing or partial output remains missing or partial.
8. On every outcome, retain state, harness-visible interaction, target-cwd/session identity, final process status, document that exists, and cleanup notes. Kill remaining processes, dismiss the completed shell record, and verify no comparison-harness shell or process remains before starting another.
7. Locate the harness-authored document if it exists without altering it. Never author, reconstruct, complete, rewrite, or improve it. Missing or partial output remains missing or partial.
8. On every outcome, kill remaining processes, dismiss the completed shell record, and verify no comparison-harness shell or process remains. Only after that process cleanup, copy the exact harness-visible transcript and any unchanged harness-authored document from the external target into controller-owned scratch evidence; retain target-cwd/session identity, final process status, and cleanup notes. Then remove that harness's external target root and verify it is gone before starting another harness.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Transcript salvage uses wrong source

Medium Severity

The new cleanup step says to copy the harness-visible transcript from the external target after dismissing the shell record. That transcript is the controller-authored disclosure record from the observed interaction, not a file in the target. Only the harness-authored document needs salvage from the external root, so this order and source can drop or miss the disclosure record.

Additional Locations (1)
Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 14deb5d. Configure here.


Do not notify completion while any harness is ready, running, or waiting. After every selected harness is finished or failed, give one aggregate notification. Review scratch assembly, then deliberately copy it to the unused immutable run identity without changing `provenance.json`, snapshots, transcripts, or harness-authored documents.

Expand Down
4 changes: 2 additions & 2 deletions TESTING_FINDINGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -477,14 +477,14 @@ Observation: the interview advanced well past the splash into a populated graph
Expected: one direct normal-width shell exposes a usable Brunch Specify exchange while preserving mission isolation and cleanup.
Disposition: pass — meets FE-1215's focused stock-Pi/direct-Brunch smoke leaf (scope card consumed on completion; status in `memory/PLAN.md` §operator-comparison-workflow).

#### CS2 · private-mission isolation · high · pass with residual risk
#### CS2 · private-mission isolation · high · pass; placement risk closed

Concern: whether the private mission leaked into target-visible input or was ingested from disk.
Evidence: transcript scan of the session/`debug/trajectory.ndjson` — zero occurrences of mission-only phrasings (`private-mission`, `top-level-session-only`, `The PM wants`, `Decision latitude`, `Conversational and disclosure posture`) and zero references to the `private-mission.md`/`harness-setup.md` paths; the one matching phrase is the mission's sanctioned natural opening. Full tool-call audit: 20 calls (12 `ask`, 4 `mutate_graph`, 2 `present_digest`, 1 `present_candidates`, 1 `read`); the sole `read` targeted `src/agents/skills/propose/SKILL.md`; no `ls`/`find`/`grep` were used.
Observation: no mission text, path, or wholesale payload entered the Brunch cwd/session; the spec content came from the elicitation exchanges, not file ingestion.
Expected: the private-mission boundary holds without a separate actor process.
Residual risk: `private-mission.md` is stored at the run root, two levels **above** the Brunch target cwd (`targets/brunch`), and the Brunch agent's active tools include generic `read`/`ls`/`find`/`grep` (per `debug/system-prompt.md`). Isolation held **behaviorally** (the agent did not traverse up), not by a filesystem jail; a differently-behaved contender or retry could `read ../../private-mission.md`.
Disposition: pass for this run; residual isolation-hardening routed to `ln-plan` as `comparison-mission-isolation-hardening` (`memory/PLAN.md`). Owner: that follow-up frontier. Re-entry trigger: any future comparison run, before treating placement isolation as guaranteed.
Disposition: closed by FE-1320. `/compare-specs` now gives each harness a fresh system-temporary external target root outside controller checkout, scratch, and retained trees; actual-entry-point tests reject the `../../private-mission.md` rival and preserve exact controller evidence through target cleanup. This closes ordinary target-relative ancestor traversal, not unrestricted absolute-path or whole-host discovery. The real Brunch + Claude conduct witness remains owned by `saved-mission-comparison-witness`.

#### CS3 · harness cleanup · medium · unverified

Expand Down
6 changes: 6 additions & 0 deletions docs/archive/PLAN_HISTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,12 @@
This file is the active POC-line plan archive for `memory/PLAN.md`.
Legacy pre-`next` history was moved out of the live docs tree with the old archived implementation.

## 2026-08-05 FE-1320 comparison mission isolation closeout

`comparison-mission-isolation-hardening` ([FE-1320](https://linear.app/hash/issue/FE-1320/structural-private-mission-isolation-for-compare-specs)) closed the CS2 placement risk without introducing a controller runtime or generic sandbox. The prompt-owned `/compare-specs` procedure now allocates each harness a fresh system-temporary external target root outside controller checkout, scratch, and retained trees; Brunch still launches from the repository root and receives the target through `--workspace`.

The actual-entry-point contract proves target-root `read`/`ls`/`find`/`grep` visibility excludes controller-private mission material, the known `../../private-mission.md` rival fails, controller snapshots and exact target-authored output survive process and target cleanup, and historical FE-1215 retained bytes remain untouched. This boundary covers ordinary target-relative ancestor traversal only; it does not claim unrestricted absolute-path or whole-host isolation. The real Brunch + Claude run and revision-immutability evidence remain owned by `saved-mission-comparison-witness`.

## 2026-08-05 FE-1318 canonical document reconciliation closeout

`canonical-document-reconciliation` closed FE-1316 rows R06, R07, and R12 without creating another planning or comparison status store. The planning-substrate assessment now records its PLAN-replacement prescription as unadopted; PLAN, SPEC, Linear/Graphite, and co-located topology retain their established authority split.
Expand Down
26 changes: 4 additions & 22 deletions memory/PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

- **Ship gate:** FE-1187 `walkthrough-remediation-2` is still the sole remaining ship-gate frontier. `remediation-4` landed the mixed-settlement review contract, populated-only Impact Ledger rendering, terminal-adaptive shell, persistent Specify elicitation style, deterministic menus, Ask mechanics, Continue-language cleanup, KA handoff, and the R6 terminal-status rail; extractor/oracle hardening and the fresh R8–R10 0/3 campaign remain paused at user direction. Execute evidence O7–O9 is still KA-owned.
- **Shared host follow-through:** FE-1200 proved the standalone web foundation, but the TUI still owns a separate live `AgentSession`, `SessionEventRelay`, and `/rpc/driver` sidecar path. The open architecture arc is `shared-session-host-convergence`.
- **Comparison lanes:** the repo keeps two distinct evaluation doors: seed-based intra-product testing and mission-driven cross-product comparison. FE-1241 closed the brownfield comparison cases; FE-1215 landed the approachable operator workflow; the remaining operator residue is the structural mission-isolation hardening plus the later real Brunch+Claude saved-mission witness.
- **Comparison lanes:** the repo keeps two distinct evaluation doors: seed-based intra-product testing and mission-driven cross-product comparison. FE-1241 closed the brownfield comparison cases; FE-1215 landed the approachable operator workflow; FE-1320 structurally separated controller-private mission material from fresh external harness targets. The remaining operator residue is the later real Brunch+Claude saved-mission witness.
- **Executor / Execute evidence queue:** several KA fronts are implementation-merged but still carry explicit outer evidence: `host-landing`, `executor-plan-synthesis`, `execution-comparison-tracer`, `executor-plan-coherence`, `executor-slice-verification-repair`, and `greenfield-secure-drop-demo`.
- **Current seams and discipline:** Brunch ships on `1.0.0-alpha.x`. D125-L's live ask registry is the structured-exchange headless surface; the transcript-backed pending projection remains compatibility-only. Sweep classification remains fail-closed on exchange-schema terminal names (D117-L); the larger capture-conditional watermark question remains A40-L. Co-located `src/**/TOPOLOGY.md` files own current topology; scratch evidence is not durable until promoted to `.fixtures/runs/`.

Expand Down Expand Up @@ -52,16 +52,15 @@

### Recently Completed

- 2026-08-05 `comparison-mission-isolation-hardening` (FE-1320) — **✓ complete:** `/compare-specs` now places every harness in a fresh system-temporary external target root outside controller checkout, scratch, and retained trees; actual-entry-point tests cover the CS2 `../../private-mission.md` rival, target-root visibility, retained exact output, and target cleanup without claiming whole-host isolation.
- 2026-08-05 `canonical-document-reconciliation` (FE-1318) — **✓ complete:** the unadopted PLAN-replacement proposal is historical, seven superseded design surfaces now point to current authority, comparison prose distinguishes four execution cases from three configured E2E contracts and one retained witness, and active document links resolve.
- 2026-08-05 `host-landing-oracle-identity` (FE-1317) — **✓ complete:** the immutable host-landing oracle pack now includes all five behavior-bearing PTY inputs, with regression coverage for one-file-at-a-time hash sensitivity and identical-input stability.
- 2026-08-06 `system-reorientation-audit` (FE-1316) — **✓ complete:** thirteen post-cleanup concerns received evidence-backed dispositions; one stale exchange-guidance defect was fixed, two bounded frontiers were admitted, and existing owners absorbed the remaining promoted work without creating a parallel queue.

Older completion history and archived completed frontier definitions live in [`docs/archive/PLAN_HISTORY.md`](../docs/archive/PLAN_HISTORY.md).

### Next

- `saved-mission-comparison-witness` — later operator-owned frontier: FE-1215 is landed, so the remaining work is scheduling the real Brunch + Claude `/compare-specs` witness, then revising/rerunning the saved mission to prove historical snapshots remain immutable. Definition below.
- `comparison-mission-isolation-hardening` — admitted from FE-1215 smoke (`TESTING_FINDINGS.md` CS2): move the private mission outside the target-reachable tree (or equivalently jail the target's file tools) so the D134-L/I67-L mission boundary is structural, not conduct-dependent. Definition below.
- `shared-session-host-tracer` — prove the one-host TUI attachment seam that FE-1200 deliberately left open. Definition below.
- `shared-session-host-cutover` — wait-gated on the tracer; close the enumerated TUI/web host surface, then delete the raw sidecar relay and `/rpc/driver`. Definition below.

Expand Down Expand Up @@ -133,22 +132,9 @@ Legacy link target; see Horizon.
- **Objective:** prove that the corrected approachable saved-mission workflow works through its actual entry point and that mission revision affects future runs without rewriting historical evidence.
- **Acceptance:** a stock Pi session runs the real `/compare-specs` prompt through ordinary-text approvals; the top-level session is the sole simulated-user actor and drives one direct Brunch shell then one direct Claude shell; the first run proves mission-consistent conduct, no mission leakage, honest outcomes/cleanup, unchanged target-authored documents, one aggregate notification, and a readable report; the operator then revises the mission and approves a second run whose edits do not rewrite the first run's retained snapshots.
- **Verification:** outer only — operator-led first run plus revision/rerun through the real stock-Pi prompt, with artifact inspection and snapshot fingerprints.
- **Dependencies:** hard on operator scheduling; FE-1215's D134-L remediation landing is already satisfied.
- **Dependencies:** hard on operator scheduling; FE-1215 direct-control remediation and FE-1320 target-placement hardening are satisfied.
- **Traceability:** req 24/A5-L; D70-L; D134-L/I67-L; FE-1210 evidence discipline; SPEC Verification Design `saved-mission-comparison-witness` gate.

### comparison-mission-isolation-hardening

- **Name:** Structural private-mission isolation for `/compare-specs`
- **Linear:** unassigned — create at pickup in Frontend / brunch.
- **Branch:** tbd at pickup; stack after the landed FE-1215 workflow when this follow-on starts.
- **Kind:** bounded hardening over the retained D134-L mission boundary.
- **Certainty:** proving.
- **Status:** admitted from `TESTING_FINDINGS.md` CS2; not started.
- **Objective:** make the private mission structurally unreachable from the comparison target by storing it outside the target-reachable tree or equivalently constraining target file tools to the harness cwd.
- **Acceptance:** the target cannot discover or read the private mission by normal `read`/`ls`/`find`/`grep` access; the run still retains the private mission for controller/operator evidence; no FE-1215 historical run is rewritten; the fix does not widen into a generic comparison-runner redesign.
- **Verification:** focused artifact-path and target-visibility tests plus one targeted smoke over the retained `/compare-specs` flow.
- **Traceability:** D134-L/I67-L; `TESTING_FINDINGS.md` CS2.

### shared-session-host-tracer

- **Name:** Shared session host tracer — one runtime authority for TUI and React
Expand Down Expand Up @@ -200,14 +186,10 @@ active:
blocked_reentry: extractor/oracle reconciliation -> fresh R8–R10 0/3 campaign -> A48-L -> consolidated outer checkpoint
cross_stream: O7/O8/O9 + carved Execute beats stay KA-owned
-[stack]-> cli-mode-entry

next:
saved-mission-comparison-witness
gated_by: operator availability
depends_on: landed FE-1215 workflow
comparison-mission-isolation-hardening
source: TESTING_FINDINGS.md CS2 from FE-1215 witness
hardens: D134-L/I67-L private-mission boundary
dependencies_satisfied: FE-1215 direct control | FE-1320 external target placement
shared-session-host-tracer
retires: A47-L
proves: one host-owned writable runtime + real TUI + React attachment
Expand Down
Loading
Loading