From f3e0cebc35947019de472731ccbd542dc30bcd14 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 28 Aug 2026 10:02:51 +0000 Subject: [PATCH 1/6] plan: mask-membrane-valhalla-integration-v1 (the layered consolidation) The PR #44->#46 arc's rulings synthesized into one wave plan under the operator's layer model: masking underneath (hop = MASK x CLASSVIEW -> MASK as the execution currency), Panama the membrane (shape/meaning/ operations cross, ownership does not), Valhalla cheap addresses (describe the one substrate, never become a second one), everything else underneath (placement, publication, worker topology, sealed horizon -- substrate-private). Seven frozen decisions cited (F1-F7) so no wave can re-litigate them, including the measured F7 ground truth: compute parallelizes 3.27x but the convergence tail is ~90% of a 64K cycle, so the write seam stays blocked with GridLake and carries no D-id. Waves ordered safety-first: W0 mechanical fences for the council- corrected doctrine (each proven able to fire); W1 close the membrane's two named gaps (epoch re-check, minors 2-4 lazy holders); W2 finish the mask layer (fused single-plane pass, sweeps over facet-major); W3 Valhalla promotions on measured evidence only; W4.1 temporal read binding via the contract's clean vocabulary. Falsification conditions for the ordering itself are pre-registered in the plan. Board hygiene same-commit: INTEGRATION_PLANS prepend, STATUS_BOARD D-LGJ-MMV-0..4 rows. --- .claude/board/INTEGRATION_PLANS.md | 27 ++ .claude/board/STATUS_BOARD.md | 20 ++ .../mask-membrane-valhalla-integration-v1.md | 243 ++++++++++++++++++ 3 files changed, 290 insertions(+) create mode 100644 .claude/plans/mask-membrane-valhalla-integration-v1.md diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index 806f532..d93bf4b 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -1,3 +1,30 @@ +## 2026-08-28 — mask-membrane-valhalla-integration-v1 (PROPOSED; the layered consolidation) + +Plan: `.claude/plans/mask-membrane-valhalla-integration-v1.md`. The +synthesis of the PR #44→#46 arc into one wave plan under the operator's +layer model: **masking underneath** (hop = MASK × CLASSVIEW → MASK, the +execution currency), **Panama the membrane** (shape/meaning/operations +cross, ownership does not), **Valhalla cheap addresses** (describe the one +substrate, never become a second one), **everything else underneath** +(placement, publication, worker topology, the sealed horizon — all +substrate-private). Seven frozen decisions cited (F1–F7), including the +measured F7 ground truth that the convergence tail, not compute, is ~90% +of a 64K cycle. + +Waves: **W0** mechanical fences for the council-corrected doctrine +(materialization-list gate, §E topology fence, SIMD-branch fence — each +proven able to fire); **W1** close the membrane's two named honesty gaps +(`ISS-LGJ-EPOCH-UNCHECKED` epoch re-check; minors 2-4 lazy holders so +`requireMinor` covers every minor); **W2** finish the mask layer (the +fused single-plane pass' ~10× lab claim verified through the real ABI; +register sweeps answering facet-major instead of −18); **W3** Valhalla +value-class promotions on measured evidence only; **W4** temporal READ +binding via the contract's clean vocabulary — the WRITE seam stays +blocked with GridLake (F5) and deliberately carries no D-id. + +**Status: PROPOSED.** D-LGJ-MMV-0 (W0 fences) is the cheapest first +action; W1.1's Phase-0 council spec is already written in-session. + ## 2026-08-17 — ogar-machine-v1 (EXPLORATORY; the population emulator) Plan: `.claude/plans/ogar-machine-v1.md`. Captured from the operator's diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index c4f3663..d56598c 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -1,3 +1,23 @@ +## mask-membrane-valhalla-integration-v1 (2026-08-28) + +Plan: `.claude/plans/mask-membrane-valhalla-integration-v1.md` — the +layered consolidation of the PR #44→#46 arc (masking underneath / Panama +the membrane / Valhalla cheap addresses / everything else underneath). + +| D-id | Deliverable | Status | +|---|---|---| +| D-LGJ-MMV-0 | Mechanical fences: materialization-list gate, §E worker-topology fence, SIMD-branch fence — each proven able to fire | Queued | +| D-LGJ-MMV-1a | Epoch re-check on the cached-descriptor path (resolves `ISS-LGJ-EPOCH-UNCHECKED`); disable-run red-then-green; overhead measured against banked benches | Queued | +| D-LGJ-MMV-1b | Minors 2-4 lazy `Downcalls` holders; `OldAbiCompatTest` minor-1 leg; doctrine "fails cleanly at the call" wording restored by storno | Queued | +| D-LGJ-MMV-2a | Fused single-plane columnar pass — the lab arm's ~10× claim verified through the real ABI, equivalence before timing | Queued | +| D-LGJ-MMV-2b | Register-sweep family (§14-16) answers facet-major; −18 refusal re-pinned two-sided, kept reachable | Queued | +| D-LGJ-MMV-3 | Valhalla value-class promotions where the three-truths lab shows a measured facade-path win; each reversible | Queued (evidence-gated) | +| D-LGJ-MMV-4 | Temporal read binding via `LanceVersion`/`VersionRange`/`TemporalPov` (contract vocabulary, never the Kanban-entangled types) | Queued (substrate-first per F6) | + +W4.2 (the write seam) deliberately has NO row — blocked with GridLake +(plan F5) and substrate-side by the F7 measurement; a row here would be +the consumer-side scheduling creep §E forbids. + ## the mask-native SWEEP arc — ABI minors 5-8 (2026-08-25) The execution half of the mask path. The BUILD half (`lgj_op_eq_classid`) diff --git a/.claude/plans/mask-membrane-valhalla-integration-v1.md b/.claude/plans/mask-membrane-valhalla-integration-v1.md new file mode 100644 index 0000000..5ac4ef0 --- /dev/null +++ b/.claude/plans/mask-membrane-valhalla-integration-v1.md @@ -0,0 +1,243 @@ +# mask-membrane-valhalla-integration-v1 — the layered integration plan + +**Status: PROPOSED (2026-08-28).** Written after the PR #44→#46 arc closed: +the mask algebra restored and made fast (minor 10 columnar store, 4.7–5.9× +through the real ABI), the simd.rs isomorphism pinned as CLAUDE.md's +ENFORCEMENT LAYER (E1–E6), the zero-copy/memory-safety doctrine pinned and +then council-corrected (5+3, PR #46), and the 64K execution end measured +in-tree (`exp-kia-a2-64k-fresh-run.txt`). This plan is the consolidation: +it names each layer's finished state, its open honesty gaps, and the wave +order that closes them — **masking underneath, Panama the membrane, +Valhalla cheap addresses, everything else underneath.** + +## The layer model (frozen — this plan implements it, never re-litigates it) + +``` +Application / TinkerPop semantic intent: where()/hop()/out()/has() + │ (vocabulary only — never execution) + ▼ +lance-graph-java (facade) ndarray's simd.rs, one level up: 37-fns- + │ 0-instructions discipline. Names things a + │ backend already does (E5). Never computes + │ (E1). Scalar only as cfg(test) oracle (E2). + ▼ +Valhalla cheap ABI-shaped ADDRESSES: FacetId, Mask + │ handle, ProjectionHint, LaneDesc-shaped + │ values. Describe/address the one native + │ substrate; never own enough state to + │ become a second one. + ▼ +Panama (internal/ffm) THE MEMBRANE. Shape crosses. Meaning + │ crosses. Operations cross. Ownership does + │ not cross. Descriptors, layout contract, + │ generation-checked handles, manifest-first + │ handshake. FFM types quarantined here +════════╪═══════ membrane ══════ (ApiSurfaceTest = this repo's simd-savant) + ▼ +lgj-abi (Rust) MASK ALGEBRA IS THE EXECUTION CURRENCY. + │ hop = MASK × CLASSVIEW/WIDEFIELDMASK → + │ MASK, word-parallel, layout-blind Java + │ above (minor 10). Bulk or lifecycle, ∝ + │ n_rows, never per-row. + ▼ +ndarray::simd backends (AVX-512/AVX2/NEON/scalar). All + │ SIMD from the polyfill, never hpc::*, + │ never raw intrinsics. Selection is + │ ndarray's business — invisible above. + ▼ +lance-graph substrate EVERYTHING ELSE UNDERNEATH: placement + (row_of(owner)-shaped landing), publication + (freeze → coalesce → SOLE writer → one + DatasetVersion), worker topology (an + internal scheduling variable, never a + consumer API), the sealed temporal horizon. +``` + +Frozen decisions this diagram carries (cited, not re-arguable here): + +- F1. The mask-native invariant + named exceptions (Import/Materialisation) + — `CLAUDE.md` § mask-native invariant, operator-ruled 2026-08-18. +- F2. The simd.rs isomorphism E1–E6 — `CLAUDE.md` § ENFORCEMENT LAYER, + operator-ruled 2026-08-27 (`E-JAVA-IS-SIMD-RS-VALHALLA-PANAMA-IS-THE- + POLYFILL-1`). +- F3. Zero-copy + memory safety, as **corrected by PR #46** — the scoped + wording is canonical (handle-mediated registry check; minors 2-4 as the + tracked `requireMinor` exception; the five-item exhaustive + materialization list; the Import carve-out on `segment.set`). +- F4. Ownership constitution + §E worker-topology rule — the operator's + enforcement pass, audited clean 2026-08-27: no `workers(`/`workerCount`/ + `parallelism(` ever becomes consumer semantics; EXP-KIA's worker sweep + stays a native benchmark independent variable. +- F5. GridLake (the generic parallel write API) stays **BLOCKED until + deterministic landing identity is independent of producer arrival order** + — `mask-native-navigation-correction-v1.md` §10; no placement algorithm + enters the Java ABI. +- F6. The missing-capability STOP rule — a needed primitive lands + substrate-first (`ndarray::simd` → lgj-abi → membrane → facade), never + hand-rolled a layer up. Three-times-proven (D-LGJ-SWEEP-5/6, W6/W7). +- F7. The measured ground truth for "everything else underneath": + compute parallelizes 3.27× @ workers=8 but the cast/collect/wal/apply + convergence tail is FLAT across worker count and is ~90% of a steady + cycle (`E-EXP-KIA-A2-64K-CONVERGENCE-TAIL-DOMINATES-1`). Any seam that + only accelerates compute optimizes the wrong ~10%. + +## Where each layer stands (measured, not asserted) + +| Layer | Done | Open (the honesty gaps, all council-named) | +|---|---|---| +| Facade | layout-blind (disable-run red at first divergent address); geometry derived from `Layouts` only (J2 closed); no compute path (G2 + reflective allowlist) | nothing structural — grows only names for W2/W4 verbs | +| Valhalla | descriptors are already ABI-shaped records (`LaneWindow`, `LgjLaneDesc` mirror); three-truths lab method stands | value-class promotion is a LAB question until measured (E4 keeps Vector API permanently lab) | +| Panama | manifest-first handshake (magic→major→minor→sizes→endianness); generation-checked handles, 23/23 registry falsifiers green; FFM quarantine reflection-enforced | **G-A**: cached-descriptor path is `closed`-boolean-guarded only — `epoch` field designed, unconsulted (`ISS-LGJ-EPOCH-UNCHECKED`). **G-B**: minors 2-4 fail at `Downcalls.`, not at the `requireMinor` call | +| Mask algebra | hop = `src ∧ class_f ∧ struct_f` word-parallel (R1); facet-match reduction native (minor 9); columnar store (minor 10) 4.7–5.9× over AoS through the real ABI | **G-C**: fused single-plane columnar pass (~10× further per the lab arm, zero new kernels). **G-D**: register-sweep family answers columnar with an honest −18, not a result | +| ndarray::simd | every kernel routes through the polyfill; backend diagnostic-only above | any new mask primitive (e.g. `mask_xor`) lands HERE first (F6) | +| Substrate | 64K compute proven parallel + digest-stable; sole-writer publication sequential-by-design | **G-E**: the convergence tail is where the time is (F7); the seam waits on the F5 landing-key gate — substrate-side work, never this repo's | + +## The waves + +Ordering rule: close the membrane's safety story first (a fast path over a +membrane with named holes is backwards), then finish the mask layer's speed +story, then let the address layer and the substrate seam follow evidence. +Every wave lands with board hygiene same-commit, disable-runs +red-then-green, and NO new consumer-facing topology/backend semantics +(F2/F4 gate every wave, mechanically). + +### W0 — mechanical fences for the doctrine (cheap, do first) + +The PR #46 corrections turned three prose claims into checkable +properties. Make them tests so they cannot silently drift again: + +- **W0.1** Materialization-list gate: a committed check (test or CI grep + with an allowlist) that fails when a sixth production + `long[]`/`copyOf`/`toArray` site appears outside the five named ones — + the enumeration went stale once inside the very tree it audited; a list + without a gate is a hand-maintained artifact with extra steps. +- **W0.2** Worker-topology fence: the §E grep + (`workers(`/`workerCount`/`parallelism(`/`threads(`/`partitions(`/ + `shards(` over `java/src/main` + ABI structs/exports) as an executable + check beside `ApiSurfaceTest`, not a session ritual. +- **W0.3** SIMD-branch fence: assert no `src/main` conditional consumes + `simdBackend()` (diagnostic-only, F2/E4). +- Gate: each fence proven able to fire (plant the violation, watch it go + red, remove it) — the falsifiability rule, applied to the fences + themselves. + +### W1 — close the membrane (G-A + G-B) + +- **W1.1 Epoch re-check** (resolves `ISS-LGJ-EPOCH-UNCHECKED`; Phase-0 + spec already written in-session, council question sets pre-registered). + Target resolution (a): compare the cached `LaneWindow.epoch` against the + live resource epoch on the access path (`Engine.epoch(handle)` exists; + a lighter epoch-only export is the fallback IF measurement shows the + full 32-byte `lgj_resource_info` read regresses the banked hop/columnar + benchmarks). Fallback (b) — formal unreachability proof + doctrine + downgrade — only on measured cost, never on assumption. + Gates: a disable-run test constructing same-slot-reuse-while-cached goes + red with the check off, green with it on; all existing + `LifetimeTest`/`RowStoreLifetimeTest` falsifiers stay green; measured + per-access overhead banked before/after. +- **W1.2 Lazy-holder migration for minors 2-4**: move the eager + `MethodHandle` resolution for row store/edges/hop into per-minor lazy + nested holders (the pattern minors 5+ already use), so an ABI-0.1 + library fails at `requireMinor(N)` with a clean message, at every minor. + Gates: `OldAbiCompatTest` gains the minors-2-4 leg it structurally could + not have before (build a real minor-1 library in a worktree beside the + siblings — /tmp worktrees break the path deps, measured); the + doctrine's original unconditional "fails cleanly at the call" wording is + RESTORED in CLAUDE.md only after this lands, as its own storno line. +- On W1 completion: the two scope-notes PR #46 added to the doctrine + become historical (marked resolved in place per append-only rules), and + the membrane's safety story is unconditional for the first time. + +### W2 — finish the mask layer (G-C + G-D) + +- **W2.1 Fused single-plane pass**: the lab arm measured ~10× further + headroom over the shipped columnar hop by fusing the classid/struct + predicates into one plane traversal — zero new kernels claimed by the + lab arm; verify that claim against the real ABI the way minor 10 was + verified (equivalence asserted BEFORE timing, pinned hop identical, + bytes-differ anti-vacuity). If a new primitive IS needed after all, it + lands in `ndarray::simd` first (F6) with backend/tail parity before any + ABI exposure. +- **W2.2 Register sweeps over facet-major**: replace the −18 + `UNSUPPORTED_LAYOUT` refusal with a real columnar answer for + `lgj_reduce_facet_sum`/`_resolved`/`lgj_row_layout_probe`. The refusal + was the honest placeholder; the lanes (33–97) already serve every field + contiguously, so the sweep becomes a contiguous-lane reduction. The −18 + path STAYS reachable for genuinely unsupported future layouts — re-pin + the two-sided test rather than deleting it. +- Gates: byte-identical results AoS vs columnar on every sweep (two + independent oracles, the minor-9 discipline); banked bench extended + with the sweep arms; ABI minor bump per additive symbol/behavior; + `abi.md` sections in the same commit. + +### W3 — Valhalla: cheap addresses, on evidence only + +- Scope: promote the descriptor vocabulary (`FacetId`, `MaskId`-shaped + handles, `ProjectionHint`, carving ordinals) to Valhalla value classes + where the lab's three-truths method shows a real win (flattening, + scalarization, allocation-free composition) — the ownership test from + the enforcement pass decides admissibility: *does the value + describe/address the one native substrate, or does Java now own enough + state to be a second substrate?* Describe → allow. Reproduce → STOP. +- Non-goals, permanent: Vector API in `src/main` (E4); any public + signature carrying FFM or `internal.*` types (E6); constrained value + types that a user can fabricate into a valid-looking native identity + (raw `long` constructors stay package-private/native-minted). +- Gates: R4/R10-style measured before/after on the real facade paths (the + substrate half — carving groups ≤4 B, 512/region/block 64-alignment — + is already pinned as tests from minor 10); JDK line pinned per + `r12-jdk-line` conventions; every promotion individually reversible. + +### W4 — everything else underneath: the substrate seam, measurement-gated + +- **W4.1 Read side (unblocked now)**: bind the sealed temporal horizon + through the membrane using the contract's CLEAN vocabulary — + `LanceVersion`/`VersionRange`/`TemporalPov` — never the engine's + Kanban-entangled types (frozen in + `mask-native-navigation-correction-v1.md` §8.1). A Java consumer asks + "as of which sealed version" as a descriptor; the substrate resolves it. + Substrate-first: whatever version-pinned read primitive lgj-abi needs + goes through F6. +- **W4.2 Write side (BLOCKED, and this plan keeps it blocked)**: the + F7 measurement says the convergence tail (cast/collect/wal/apply, + sequential by design, sole writer) is ~90% of the cycle — so the + valuable seam work is on the SUBSTRATE side of the membrane + (lance-graph's landing-key gate, F5), not here. lance-graph-java's + whole write story remains: stage semantic intent, cross as descriptors, + the sole native writer lands and publishes. No batch/commit verbs enter + the facade until F5's gate resolves upstream; when it does, the verbs + arrive as names for what the substrate then already does (E5). +- Gate for W4.1: crossing count ∝ queries not rows; a version-pinned read + proven byte-stable against a later write (the time-machine falsifier); + zero new Java-side state beyond the descriptor. + +## D-ids (STATUS_BOARD rows land with this plan's PR) + +| D-id | Deliverable | Wave | +|---|---|---| +| D-LGJ-MMV-0 | The three mechanical fences, each proven able to fire | W0 | +| D-LGJ-MMV-1a | Epoch re-check wired (or measured-and-proven fallback), `ISS-LGJ-EPOCH-UNCHECKED` resolved | W1 | +| D-LGJ-MMV-1b | Minors 2-4 lazy holders + `OldAbiCompatTest` minor-1 leg; doctrine wording restored by storno | W1 | +| D-LGJ-MMV-2a | Fused single-plane columnar pass, equivalence-then-timing | W2 | +| D-LGJ-MMV-2b | Register sweeps answer facet-major; −18 re-pinned two-sided | W2 | +| D-LGJ-MMV-3 | Valhalla value-class promotions, individually measured + reversible | W3 | +| D-LGJ-MMV-4 | Temporal read binding via contract vocabulary | W4.1 | + +W4.2 deliberately has no D-id: it is not this repo's work until F5 +resolves upstream, and giving it a row here would invite exactly the +consumer-side scheduling creep F4 forbids. + +## What would falsify this plan's ordering + +- If W1.1's measurement shows the epoch re-check costs more than the + fused pass gains (unlikely — one u64 compare vs a plane traversal), the + W1/W2 order flips for the affected path only, recorded as a dated + amendment here, not silently. +- If the W2.1 lab claim (~10×, zero new kernels) does not survive + real-ABI verification, W2.1 re-enters through F6 as an ndarray::simd + primitive proposal — the plan does not pre-authorize a membrane-side + workaround. +- If Valhalla's measured wins in W3 are nil on the real facade paths, W3 + closes as "lab-confirmed, production-deferred" — a valid outcome, not a + failure; the addresses are already cheap enough to have hit every + allocation gate. From 1ed338de0c3575e9b6efaa8b8529962610615d3f Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 28 Aug 2026 10:35:18 +0000 Subject: [PATCH 2/6] plan: apply the three Codex P2 corrections 1. The measured range now includes its slowest arm: 3.8-5.9x (3.8 all-rows / 4.7 classid / 5.9 hop2) -- stating 4.7-5.9x silently excluded the all-rows measurement, against the plan's own measure-then-pin premise. 2. W1.1's overhead gate now targets the cached-descriptor accessors THEMSELVES (classidAt/payload accessors via lane(), Mask's cached words) -- the banked hop/columnar benches run entirely through native operations and are structurally blind to per-access epoch overhead; they stay only as a regression backstop. 3. The D-LGJ-MMV-1a status row now names BOTH accepted outcomes (wire the check, or the falsifier-backed unreachability proof) so the documented fallback can truthfully complete the row. --- .claude/board/STATUS_BOARD.md | 2 +- .../mask-membrane-valhalla-integration-v1.md | 20 ++++++++++++++----- 2 files changed, 16 insertions(+), 6 deletions(-) diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index d56598c..cfe5745 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -7,7 +7,7 @@ the membrane / Valhalla cheap addresses / everything else underneath). | D-id | Deliverable | Status | |---|---|---| | D-LGJ-MMV-0 | Mechanical fences: materialization-list gate, §E worker-topology fence, SIMD-branch fence — each proven able to fire | Queued | -| D-LGJ-MMV-1a | Epoch re-check on the cached-descriptor path (resolves `ISS-LGJ-EPOCH-UNCHECKED`); disable-run red-then-green; overhead measured against banked benches | Queued | +| D-LGJ-MMV-1a | Resolve `ISS-LGJ-EPOCH-UNCHECKED` by EITHER accepted outcome: (a) epoch re-check wired on the cached-descriptor path, disable-run red-then-green, per-accessor overhead measured on the accessors themselves (not the native benches — they never touch this path); or (b) a falsifier-backed unreachability proof + doctrine downgrade, chosen only on measured cost | Queued | | D-LGJ-MMV-1b | Minors 2-4 lazy `Downcalls` holders; `OldAbiCompatTest` minor-1 leg; doctrine "fails cleanly at the call" wording restored by storno | Queued | | D-LGJ-MMV-2a | Fused single-plane columnar pass — the lab arm's ~10× claim verified through the real ABI, equivalence before timing | Queued | | D-LGJ-MMV-2b | Register-sweep family (§14-16) answers facet-major; −18 refusal re-pinned two-sided, kept reachable | Queued | diff --git a/.claude/plans/mask-membrane-valhalla-integration-v1.md b/.claude/plans/mask-membrane-valhalla-integration-v1.md index 5ac4ef0..616fb1d 100644 --- a/.claude/plans/mask-membrane-valhalla-integration-v1.md +++ b/.claude/plans/mask-membrane-valhalla-integration-v1.md @@ -1,7 +1,7 @@ # mask-membrane-valhalla-integration-v1 — the layered integration plan **Status: PROPOSED (2026-08-28).** Written after the PR #44→#46 arc closed: -the mask algebra restored and made fast (minor 10 columnar store, 4.7–5.9× +the mask algebra restored and made fast (minor 10 columnar store, 3.8–5.9× through the real ABI), the simd.rs isomorphism pinned as CLAUDE.md's ENFORCEMENT LAYER (E1–E6), the zero-copy/memory-safety doctrine pinned and then council-corrected (5+3, PR #46), and the 64K execution end measured @@ -88,7 +88,7 @@ Frozen decisions this diagram carries (cited, not re-arguable here): | Facade | layout-blind (disable-run red at first divergent address); geometry derived from `Layouts` only (J2 closed); no compute path (G2 + reflective allowlist) | nothing structural — grows only names for W2/W4 verbs | | Valhalla | descriptors are already ABI-shaped records (`LaneWindow`, `LgjLaneDesc` mirror); three-truths lab method stands | value-class promotion is a LAB question until measured (E4 keeps Vector API permanently lab) | | Panama | manifest-first handshake (magic→major→minor→sizes→endianness); generation-checked handles, 23/23 registry falsifiers green; FFM quarantine reflection-enforced | **G-A**: cached-descriptor path is `closed`-boolean-guarded only — `epoch` field designed, unconsulted (`ISS-LGJ-EPOCH-UNCHECKED`). **G-B**: minors 2-4 fail at `Downcalls.`, not at the `requireMinor` call | -| Mask algebra | hop = `src ∧ class_f ∧ struct_f` word-parallel (R1); facet-match reduction native (minor 9); columnar store (minor 10) 4.7–5.9× over AoS through the real ABI | **G-C**: fused single-plane columnar pass (~10× further per the lab arm, zero new kernels). **G-D**: register-sweep family answers columnar with an honest −18, not a result | +| Mask algebra | hop = `src ∧ class_f ∧ struct_f` word-parallel (R1); facet-match reduction native (minor 9); columnar store (minor 10) 3.8–5.9× over AoS through the real ABI (3.8× all-rows / 4.7× classid / 5.9× hop2 — the full measured range, slowest arm included) | **G-C**: fused single-plane columnar pass (~10× further per the lab arm, zero new kernels). **G-D**: register-sweep family answers columnar with an honest −18, not a result | | ndarray::simd | every kernel routes through the polyfill; backend diagnostic-only above | any new mask primitive (e.g. `mask_xor`) lands HERE first (F6) | | Substrate | 64K compute proven parallel + digest-stable; sole-writer publication sequential-by-design | **G-E**: the convergence tail is where the time is (F7); the seam waits on the F5 landing-key gate — substrate-side work, never this repo's | @@ -128,13 +128,23 @@ properties. Make them tests so they cannot silently drift again: Target resolution (a): compare the cached `LaneWindow.epoch` against the live resource epoch on the access path (`Engine.epoch(handle)` exists; a lighter epoch-only export is the fallback IF measurement shows the - full 32-byte `lgj_resource_info` read regresses the banked hop/columnar - benchmarks). Fallback (b) — formal unreachability proof + doctrine + full 32-byte `lgj_resource_info` read is too costly per access). + Fallback (b) — formal unreachability proof + doctrine downgrade — only on measured cost, never on assumption. + **The measurement target is the cached-descriptor accessors THEMSELVES** + (`RowStore.classidAt`/`payloadLow64At`/`payloadHi32At` through + `lane()`, `Mask`'s cached-`words` reads) — a Codex review correction + (PR #47): the banked hop/columnar benchmarks run entirely through + native operations and never touch the cached-descriptor path, so they + are structurally blind to per-access epoch overhead and would stay + unchanged even if every accessor gained a native crossing. A + before/after micro-measure of the actual accessors is the gate; the + banked native benches remain only a regression backstop for the + operations they do exercise. Gates: a disable-run test constructing same-slot-reuse-while-cached goes red with the check off, green with it on; all existing `LifetimeTest`/`RowStoreLifetimeTest` falsifiers stay green; measured - per-access overhead banked before/after. + per-accessor overhead banked before/after. - **W1.2 Lazy-holder migration for minors 2-4**: move the eager `MethodHandle` resolution for row store/edges/hop into per-minor lazy nested holders (the pattern minors 5+ already use), so an ABI-0.1 From 0ffd677733c2e75ecb625ee8cdd18803dfcbabf0 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 28 Aug 2026 10:38:44 +0000 Subject: [PATCH 3/6] plan: apply the five CodeRabbit findings 1. The W1.1 Phase-0 council spec is now COMMITTED (.claude/plans/epoch-recheck-phase0-v1.md) instead of "written in-session" -- a spec that exists only in a transcript is not a spec. 2. The epoch-only fallback export is declared a full ABI citizen: its own minor bump, manifest + abi.md entry, requireMinor at the call site, and an old-library rejection leg -- or the fallback recreates the missing-symbol failure class W1 exists to remove. 3. W1.2 now states the structural invariant (requireMinor BEFORE the first lazy-holder reference) and requires one independent negative probe per minor -- the existing minor-4 gate rides Mask.minus and cannot discriminate alone; an unprobed minor may not be claimed. 4. The W1-completion "unconditional" paragraph is scoped to resolution (a); under fallback (b) the issue closes DOWNGRADED-DOCUMENTED, the scope-note stays live, and no unconditional claim is made. 5. The W1/W2 flip condition gets reproducible units: same 65,536-row fixture, ns/accessor-call over >=1M calls median-of-5, threshold 2x accessor baseline; W2 proceeds on its own native paths regardless, but nothing reading through a cached descriptor ships an unconditional-safety claim while W1.1 is unresolved. Plus the MD040 language tag on the layer diagram fence. --- .claude/board/INTEGRATION_PLANS.md | 5 +- .claude/plans/epoch-recheck-phase0-v1.md | 130 ++++++++++++++++++ .../mask-membrane-valhalla-integration-v1.md | 65 ++++++--- 3 files changed, 182 insertions(+), 18 deletions(-) create mode 100644 .claude/plans/epoch-recheck-phase0-v1.md diff --git a/.claude/board/INTEGRATION_PLANS.md b/.claude/board/INTEGRATION_PLANS.md index d93bf4b..358c7b1 100644 --- a/.claude/board/INTEGRATION_PLANS.md +++ b/.claude/board/INTEGRATION_PLANS.md @@ -23,7 +23,10 @@ binding via the contract's clean vocabulary — the WRITE seam stays blocked with GridLake (F5) and deliberately carries no D-id. **Status: PROPOSED.** D-LGJ-MMV-0 (W0 fences) is the cheapest first -action; W1.1's Phase-0 council spec is already written in-session. +action; W1.1's Phase-0 council spec is COMMITTED at +`.claude/plans/epoch-recheck-phase0-v1.md` (frozen decisions, both +accepted resolutions, pre-registered gates, five per-savant question +sets). ## 2026-08-17 — ogar-machine-v1 (EXPLORATORY; the population emulator) diff --git a/.claude/plans/epoch-recheck-phase0-v1.md b/.claude/plans/epoch-recheck-phase0-v1.md new file mode 100644 index 0000000..31d9d89 --- /dev/null +++ b/.claude/plans/epoch-recheck-phase0-v1.md @@ -0,0 +1,130 @@ +# epoch-recheck-phase0-v1 — the W1.1 council SPEC (Phase 0, 5+3 protocol) + +**Status: SPEC READY (2026-08-28) — Phase 1 (the 5 savants) not yet cast.** +This is the Phase-0 specification for resolving `ISS-LGJ-EPOCH-UNCHECKED` +(`.claude/board/ISSUES.md`), referenced by +`mask-membrane-valhalla-integration-v1.md` W1.1. It was drafted in-session +on 2026-08-28 under the `/5plus3` protocol and is committed here so the +reference is durable and the question sets are auditable — a spec that +exists only in a session transcript is not a spec +(the falsifiability rule's "a doc-comment claim is not a behaviour", +applied to plans). + +## Scope finding (measured before this spec was written) + +The epoch-recheck machinery already exists on BOTH sides, more than the +issue implied: `registry.rs` stamps a *global* monotonic `epoch: u64` +(`next_epoch()`) on every resource at creation — separate from the +per-slot `generation` the handle registry checks — so two different +resources, even in the same slot, are always distinguishable +(`epochs_are_never_reused`, `registry.rs`). `Engine.java` already has +`Engine.epoch(long handle)`, which calls `lgj_resource_info` fresh and +returns the live epoch. The gap is narrower than "wire epoch checking +from scratch": **no call site in `RowStore.java`/`Mask.java` ever calls +`Engine.epoch(handle)` again and compares it against the `epoch` cached +in `LaneWindow` before trusting a cached `lanes[]`/`words` segment.** + +## 1. FROZEN DECISIONS + +- D1. `ISS-LGJ-EPOCH-UNCHECKED` — the problem statement; not re-litigated. +- D2. `CLAUDE.md`'s corrected "Pointer value is not provenance" bullet + (PR #46) — scopes the generation-registry guarantee to handle-mediated + ops and explicitly excludes the cached-descriptor path; this spec's job + is to close that exclusion or formally ratify it. +- D3. The zero-copy law: temporary scratch allowed, a second canonical + copy is not — a fix must not introduce a copy to solve staleness. +- D4. `LifetimeTest`/`RowStoreLifetimeTest`'s existing falsifiers are + frozen baselines — no regression. + +## 2. INPUT INVENTORY + +- `native/lgj-abi/src/registry.rs`: `epoch: u64` field (~:139-141), + `next_epoch()` (:233), stamped at each creation site + (:352/:365/:417/:456), tests `lane_epoch_matches_resource_epoch` + + `epochs_are_never_reused` (both green). +- `native/lgj-abi/src/exports.rs`: `epoch: entry.epoch` copied into + `LgjLaneDesc`/`LgjMaskDesc` on describe (:304, :393). +- `java/.../internal/ffm/Engine.java`: `rowCount`/`epoch(long handle)` + (:77-90, the live re-fetch primitive), `lane(...)` construction caching + `epoch` into `LaneWindow` once (:362-390), `LaneWindow` record (:405, + `epoch` field with no consumer). +- `java/.../internal/ffm/Layouts.java`: `LANE_EPOCH` (:209), + `INFO_EPOCH` (:228) — wired for reads, neither consumed. +- `java/.../RowStore.java`: `lane(int laneId)` lazy resolve + cache; the + per-row accessors reading through it; `checkedRow(long)`. +- `java/.../Mask.java`: the cached `words` lane, same shape. +- `java/.../RowStoreLifetimeTest.java`, `LifetimeTest.java` — falsifiers + to extend, not replace. + +## 3. THE PROPOSED RESOLUTION + +Two candidate resolutions; the spec commits to attempting (a) first, (b) +only if (a) is measured to be wrong: + +**(a) Wire the check.** On each access through a cached +`LaneWindow`/`words` segment, compare the cached `epoch` against a +freshly-fetched live epoch (`Engine.epoch(handle)` exists; a lighter +epoch-only export is the fallback IF measurement shows the full 32-byte +`lgj_resource_info` read is too costly per access). Mismatch → the same +`ClosedResourceException`-shaped failure the `closed` boolean already +produces — a strictly-additive second guard, not a new exception type. +**If the epoch-only export is minted, it is a full ABI citizen** (a +CodeRabbit review addition, PR #47): its own ABI minor bump, manifest + +`abi.md` entry, `requireMinor(N)` gating at the Java call site, and an +old-library rejection leg in `OldAbiCompatTest` — otherwise the fallback +recreates the missing-symbol failure class W1 exists to remove. + +**(b) Prove permanent unreachability, formally, and downgrade the +doctrine wording** — only if (a)'s cost is measured unacceptable AND a +falsifier-backed proof (a test that tries to construct the bad sequence +and fails, not prose) shows the facade's lifecycle discipline makes +slot-reuse-while-cached structurally impossible. **Outcome (b) is a +DOWNGRADE, never "resolved"** (CodeRabbit, PR #47): the doctrine's +scope-note stays live, `ISS-LGJ-EPOCH-UNCHECKED` closes as +DOWNGRADED-DOCUMENTED (a distinct status), and the unconditional-wording +restoration in `CLAUDE.md` never happens under (b). + +## 4. NON-GOALS + +- Not touching the native `epoch`/`generation` machinery (already + correct and tested — the Scope finding above). +- Not redesigning `LaneWindow`/`RowStore` caching wholesale — the fix is + a recheck at the access boundary. +- Not resolving the doctrine-text PRs — separate, already handled. + +## 5. PRE-REGISTERED GATES + +- A disable-run: with the recheck removed, a test constructing + same-slot-resource-reuse-while-cached must go RED (the check is + load-bearing, not decorative) — this test is itself a deliverable. +- With the check active: same test GREEN; every existing + `LifetimeTest`/`RowStoreLifetimeTest` falsifier stays green. +- **The overhead measurement targets the cached-descriptor accessors + THEMSELVES** (`classidAt`/`payloadLow64At`/`payloadHi32At` through + `lane()`, `Mask`'s cached-`words` reads) — a Codex review correction + (PR #47): the banked hop/columnar benches run entirely through native + operations, never touch this path, and are structurally blind to + per-access overhead. Protocol: the 65,536-row fixture, ns/accessor-call + over ≥1M calls, median of 5 runs, before/after, banked. +- Threshold (the reproducible units CodeRabbit asked for): if the + re-check cannot be brought under 2× the accessor's measured baseline + cost — including via the epoch-only export — resolution (b)'s track + opens; below 2×, (a) ships without further debate. +- `cargo test -p lgj-abi` + full Java suite green; clippy `-D warnings`. +- Board: `ISS-LGJ-EPOCH-UNCHECKED` flips OPEN → RESOLVED under (a), or + OPEN → DOWNGRADED-DOCUMENTED under (b), same commit as the code. + +## 6. PER-SAVANT QUESTION SETS (Phase 1, when cast) + +| # | lens | card | questions | +|---|---|---|---| +| 1 | handle safety | `handle-lifecycle-auditor` | Is `Engine.epoch(handle)` itself generation-checked (does its `lgj_resource_info` call go through `resolve`)? Does the recheck introduce a new TOCTOU between the epoch fetch and the subsequent read? | +| 2 | ABI membrane | `abi-membrane-warden` | Does an epoch-only export need a minor bump (yes per the resolution text — verify nothing cheaper suffices)? Does it fit the bulk/lifecycle taxonomy (`abi.md` §6)? | +| 3 | zero-copy law | `zero-copy-warden` | Is the recheck O(1) per access with zero row-proportional allocation? Does caching the comparison RESULT risk becoming a second staleness authority? | +| 4 | Java surface | `java-surface-warden` | Does the fix stay entirely inside `internal.ffm`/`RowStore`/`Mask` — no new public API, no freshness method leaking implementation physics? | +| 5 | measurement | `measurement-skeptic` | Is the disable-run's red genuinely caused by the removed check? Is the per-accessor measurement isolated and comparable (the protocol in §5), not an isolated-stage percentage? | + +Output contract: ≤10 findings each, verdict vocab +`CONFIRMS / VIOLATES / GAP / PRIOR-ART-AT / RISK`, file:line evidence, +≤2 sentences. Phase 2 consolidates before any reviewer exists; Phase 3 +reviewers see draft v2 only. diff --git a/.claude/plans/mask-membrane-valhalla-integration-v1.md b/.claude/plans/mask-membrane-valhalla-integration-v1.md index 616fb1d..97b566f 100644 --- a/.claude/plans/mask-membrane-valhalla-integration-v1.md +++ b/.claude/plans/mask-membrane-valhalla-integration-v1.md @@ -12,7 +12,7 @@ Valhalla cheap addresses, everything else underneath.** ## The layer model (frozen — this plan implements it, never re-litigates it) -``` +```text Application / TinkerPop semantic intent: where()/hop()/out()/has() │ (vocabulary only — never execution) ▼ @@ -123,14 +123,23 @@ properties. Make them tests so they cannot silently drift again: ### W1 — close the membrane (G-A + G-B) -- **W1.1 Epoch re-check** (resolves `ISS-LGJ-EPOCH-UNCHECKED`; Phase-0 - spec already written in-session, council question sets pre-registered). +- **W1.1 Epoch re-check** (the Phase-0 council spec is COMMITTED at + `.claude/plans/epoch-recheck-phase0-v1.md` — frozen decisions, input + inventory, both accepted resolutions, pre-registered gates, and the + five per-savant question sets live there, not in a session transcript). Target resolution (a): compare the cached `LaneWindow.epoch` against the live resource epoch on the access path (`Engine.epoch(handle)` exists; a lighter epoch-only export is the fallback IF measurement shows the - full 32-byte `lgj_resource_info` read is too costly per access). + full 32-byte `lgj_resource_info` read is too costly per access — and if + that export is minted it is a **full ABI citizen**: its own minor bump, + manifest + `abi.md` entry, `requireMinor(N)` at the Java call site, and + an old-library rejection leg in `OldAbiCompatTest`, or the fallback + recreates the missing-symbol failure class W1 exists to remove). + Resolution (a) resolves `ISS-LGJ-EPOCH-UNCHECKED`. Fallback (b) — formal unreachability proof + doctrine - downgrade — only on measured cost, never on assumption. + downgrade — only on measured cost, never on assumption; under (b) the + issue closes as **DOWNGRADED-DOCUMENTED, never "resolved"**, and the + doctrine's scope-note stays live (see the W1-completion paragraph). **The measurement target is the cached-descriptor accessors THEMSELVES** (`RowStore.classidAt`/`payloadLow64At`/`payloadHi32At` through `lane()`, `Mask`'s cached-`words` reads) — a Codex review correction @@ -149,14 +158,27 @@ properties. Make them tests so they cannot silently drift again: `MethodHandle` resolution for row store/edges/hop into per-minor lazy nested holders (the pattern minors 5+ already use), so an ABI-0.1 library fails at `requireMinor(N)` with a clean message, at every minor. - Gates: `OldAbiCompatTest` gains the minors-2-4 leg it structurally could - not have before (build a real minor-1 library in a worktree beside the - siblings — /tmp worktrees break the path deps, measured); the - doctrine's original unconditional "fails cleanly at the call" wording is - RESTORED in CLAUDE.md only after this lands, as its own storno line. -- On W1 completion: the two scope-notes PR #46 added to the doctrine - become historical (marked resolved in place per append-only rules), and - the membrane's safety story is unconditional for the first time. + **The structural requirement, stated as the invariant** (a CodeRabbit + addition, PR #47): every public path entering a minor-N capability + calls `Abi.requireMinor(N)` BEFORE its first reference to that minor's + lazy holder — a holder touched first re-creates the clinit failure one + step later. + Gates: `OldAbiCompatTest` gains the minors-2-4 legs it structurally + could not have before (build a real minor-1 library in a worktree + beside the siblings — /tmp worktrees break the path deps, measured), + with **one independent negative probe per minor** — the existing + minor-4 gate rides `Mask.minus` and cannot discriminate alone, so + either each minor gets its own probe or the "at every minor" claim is + narrowed to the minors actually probed; an unprobed minor may not be + claimed covered. +- **On W1 completion under resolution (a) for W1.1**: the two scope-notes + PR #46 added to the doctrine become historical (marked resolved in + place per append-only rules), and the membrane's safety story is + unconditional for the first time. **Under fallback (b) this paragraph + does NOT apply**: the doctrine keeps its scoped wording permanently, + the cached-descriptor scope-note stays live, and no "unconditional" + claim is made anywhere — a measured downgrade is a documented boundary, + not a closed gap. ### W2 — finish the mask layer (G-C + G-D) @@ -239,10 +261,19 @@ consumer-side scheduling creep F4 forbids. ## What would falsify this plan's ordering -- If W1.1's measurement shows the epoch re-check costs more than the - fused pass gains (unlikely — one u64 compare vs a plane traversal), the - W1/W2 order flips for the affected path only, recorded as a dated - amendment here, not silently. +- The W1/W2 flip condition, in reproducible units (a CodeRabbit + correction — "costs more than the fused pass gains" compared two + different denominators): on the SAME 65,536-row fixture, W1.1's number + is ns/accessor-call over ≥1M calls on the cached-descriptor accessors, + median of 5; the flip fires iff the epoch re-check cannot be brought + under **2× the accessor's measured baseline** even via the epoch-only + export. In that case W1.1 pauses on its (a)/(b) decision while W2.1 + proceeds — for its OWN paths only: W2's fused pass and sweeps run + through native operations, not the cached-descriptor accessors, so + they may be investigated and shipped regardless; nothing that READS + through a cached descriptor ships an "unconditional safety" claim + while W1.1 is unresolved. Recorded as a dated amendment here, not + silently. - If the W2.1 lab claim (~10×, zero new kernels) does not survive real-ABI verification, W2.1 re-enters through F6 as an ndarray::simd primitive proposal — the plan does not pre-authorize a membrane-side From 30986b46a5d6cf78a65bc3b008d29551ed4f640e Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 28 Aug 2026 10:54:04 +0000 Subject: [PATCH 4/6] plan: W0 scope honesty + the standing gate-observability question MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two paragraphs distilled from the PR #48 review round: - W0: lexical fences are tripwires, not proofs — the six evasion findings were one finding in six costumes. Where the property is visible in the compiled surface the fence gets a reflective arm (fence 2b, shipped on PR #48); where it exists only in source, lexical + census pins is the honest ceiling. - Falsification section: the standing question every future gate must answer in its own description — can it observe the violation it names, through the path a violation would take? Three-for-three this round: unmeasurable benchmarks, vacuous falsifiers, evadable patterns. --- .../mask-membrane-valhalla-integration-v1.md | 22 +++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/.claude/plans/mask-membrane-valhalla-integration-v1.md b/.claude/plans/mask-membrane-valhalla-integration-v1.md index 97b566f..a8c439a 100644 --- a/.claude/plans/mask-membrane-valhalla-integration-v1.md +++ b/.claude/plans/mask-membrane-valhalla-integration-v1.md @@ -121,6 +121,20 @@ properties. Make them tests so they cannot silently drift again: red, remove it) — the falsifiability rule, applied to the fences themselves. +**Scope honesty (added after the PR #48 review round):** the lexical +fences are TRIPWIRES, not proofs. Every evasion finding the reviews +produced (Codex ×3, CodeRabbit ×3) was one finding in six costumes — a +text pattern evaded by a legal spelling it did not anticipate. What the +fences actually guarantee: the accidental violation and the lazy evasion +fail loudly, and the census pins force a determined evasion to touch the +pin table in the same diff, which makes it reviewable. Where the fenced +property is visible in the COMPILED surface, the fence gets a reflective +arm instead (fence 2b: no topology-named public method — proven +non-redundant by a name/paren split across two lines that only the +reflective arm caught); where it exists only in source (call sites, +branches), lexical + census is the honest ceiling, and review + wording +discipline remain the last layer. + ### W1 — close the membrane (G-A + G-B) - **W1.1 Epoch re-check** (the Phase-0 council spec is COMMITTED at @@ -282,3 +296,11 @@ consumer-side scheduling creep F4 forbids. closes as "lab-confirmed, production-deferred" — a valid outcome, not a failure; the addresses are already cheap enough to have hit every allocation gate. + +**Standing review question for every future gate in this plan** (the +review round's recurring failure class, three-for-three: benchmarks that +structurally could not see per-accessor overhead, vacuous falsifiers, +whitespace-evadable fences): *can this gate observe the violation it +names, through the path a violation would actually take?* A gate that +cannot is worse than none — it certifies. Every new gate answers this +question in its own description before it counts as a gate. From e52bc3ae75a26bd73bcb22aeff0064fac3a47653 Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 28 Aug 2026 11:02:05 +0000 Subject: [PATCH 5/6] plan: check-then-read atomicity is a mandatory W1.1 decision output MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CodeRabbit's Major on the epoch re-check design, verified real and absorbed as a council constraint rather than deferred: the epoch fetch and the cached LaneWindow/words read are two steps, so a cross-thread close (or same-slot reuse) between them defeats the check. Two honest frames recorded with it: the race is PRE-EXISTING (the closed-boolean guard carries the identical window — the re-check neither introduces nor widens it), and it is cross-thread-only (unreachable within one thread). The Phase-0 spec now requires the council to choose, as an explicit output: (i) serialize/lease covering both steps + a mandatory interleaving falsifier, or (ii) a written scoped contract with doctrine wording matching that scope — never (ii)'s semantics under (i)'s wording, the exact overclaim class PR #46 corrected. Savant #1's question set upgraded accordingly; W1.1 in the integration plan mirrors the constraint and scopes what 'resolved' means to the chosen arm. --- .claude/plans/epoch-recheck-phase0-v1.md | 22 ++++++++++++++++++- .../mask-membrane-valhalla-integration-v1.md | 10 ++++++++- 2 files changed, 30 insertions(+), 2 deletions(-) diff --git a/.claude/plans/epoch-recheck-phase0-v1.md b/.claude/plans/epoch-recheck-phase0-v1.md index 31d9d89..923180b 100644 --- a/.claude/plans/epoch-recheck-phase0-v1.md +++ b/.claude/plans/epoch-recheck-phase0-v1.md @@ -74,6 +74,26 @@ CodeRabbit review addition, PR #47): its own ABI minor bump, manifest + old-library rejection leg in `OldAbiCompatTest` — otherwise the fallback recreates the missing-symbol failure class W1 exists to remove. +**Atomicity constraint (a second CodeRabbit review addition, PR #47 — +the check-then-read TOCTOU):** the epoch fetch and the cached +`LaneWindow`/`words` read are TWO steps; a `close(handle)` between them +(or same-slot reuse) defeats the check. Two facts frame the resolution +honestly: (1) the CURRENT `closed`-boolean guard carries the IDENTICAL +race — the epoch re-check neither introduces nor widens it; (2) within +one thread the window is unreachable (user code cannot call `close` +inside an accessor), so the residual window is exclusively +cross-thread lifecycle-vs-access. The council must therefore decide the +CONCURRENCY CONTRACT the check ships under, as an explicit output, one +of: (i) serialize epoch-fetch + cached read against close (a +read/close lock or a native lifetime lease covering both steps) — then +an INTERLEAVING falsifier for both cached paths is mandatory; or (ii) +scope the guarantee in writing to stale-cache detection BETWEEN +top-level calls, with the facade's lifecycle-vs-access thread contract +stated in `RowStore`/`Mask` javadoc and the doctrine wording matching +that scope (never "unconditional"). Silently shipping (ii)'s semantics +under (i)'s wording is the overclaim class the PR #46 council already +corrected once — not again. + **(b) Prove permanent unreachability, formally, and downgrade the doctrine wording** — only if (a)'s cost is measured unacceptable AND a falsifier-backed proof (a test that tries to construct the bad sequence @@ -118,7 +138,7 @@ restoration in `CLAUDE.md` never happens under (b). | # | lens | card | questions | |---|---|---|---| -| 1 | handle safety | `handle-lifecycle-auditor` | Is `Engine.epoch(handle)` itself generation-checked (does its `lgj_resource_info` call go through `resolve`)? Does the recheck introduce a new TOCTOU between the epoch fetch and the subsequent read? | +| 1 | handle safety | `handle-lifecycle-auditor` | Is `Engine.epoch(handle)` itself generation-checked (does its `lgj_resource_info` call go through `resolve`)? Which arm of §3's atomicity constraint — serialize/lease (i) or scoped-contract (ii) — is right, and why? (This is a mandatory decision output, not just a question — see §3; the TOCTOU is pre-existing, shared with the `closed` boolean, and cross-thread-only.) | | 2 | ABI membrane | `abi-membrane-warden` | Does an epoch-only export need a minor bump (yes per the resolution text — verify nothing cheaper suffices)? Does it fit the bulk/lifecycle taxonomy (`abi.md` §6)? | | 3 | zero-copy law | `zero-copy-warden` | Is the recheck O(1) per access with zero row-proportional allocation? Does caching the comparison RESULT risk becoming a second staleness authority? | | 4 | Java surface | `java-surface-warden` | Does the fix stay entirely inside `internal.ffm`/`RowStore`/`Mask` — no new public API, no freshness method leaking implementation physics? | diff --git a/.claude/plans/mask-membrane-valhalla-integration-v1.md b/.claude/plans/mask-membrane-valhalla-integration-v1.md index a8c439a..55cee37 100644 --- a/.claude/plans/mask-membrane-valhalla-integration-v1.md +++ b/.claude/plans/mask-membrane-valhalla-integration-v1.md @@ -149,7 +149,15 @@ discipline remain the last layer. manifest + `abi.md` entry, `requireMinor(N)` at the Java call site, and an old-library rejection leg in `OldAbiCompatTest`, or the fallback recreates the missing-symbol failure class W1 exists to remove). - Resolution (a) resolves `ISS-LGJ-EPOCH-UNCHECKED`. + The epoch fetch and the cached read are two steps — the council's + spec carries the check-then-read ATOMICITY constraint as a mandatory + decision output (serialize/lease with an interleaving falsifier, or a + written scoped contract; the spec's §3 "Atomicity constraint" is + authoritative — a CodeRabbit review addition, and the race is + pre-existing in the `closed`-boolean guard, cross-thread-only). + Resolution (a) resolves `ISS-LGJ-EPOCH-UNCHECKED` at the scope the + chosen atomicity arm actually delivers — the doctrine wording matches + that scope, never exceeds it. Fallback (b) — formal unreachability proof + doctrine downgrade — only on measured cost, never on assumption; under (b) the issue closes as **DOWNGRADED-DOCUMENTED, never "resolved"**, and the From 586d0811af6924ded317e90e2cf8b0c5508337fa Mon Sep 17 00:00:00 2001 From: Claude Date: Fri, 28 Aug 2026 11:03:52 +0000 Subject: [PATCH 6/6] plan: measurement runs under the chosen atomicity arm + knowledge: GitHub access paths MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Impact pass for the review round's findings on the plan itself: - The 2x flip gate measured 'the epoch re-check' but the atomicity decision (spec §3) changes WHAT the re-check is — a lock/lease per access under arm (i) is a categorically different cost than a bare compare. Measuring the compare and shipping the lease would leave the gate unable to observe the cost it names — the plan's own standing gate-observability question, applied to itself. Order fixed in both files: atomicity decision first, then the measurement, of that arm; the flip firing under (i) is named a valid outcome. - .claude/knowledge/github-access-paths.md: the measured three-paths / two-identities map (MCP and GH_TOKEN share user 200276742 — swapping transports never escapes a rate limit; the session proxy carries its own credential, REST-writes-capable, GraphQL pinned-set-only with resolveReviewThread excluded), the secondary-limit signature (5000/5000 remaining while GraphQL says exceeded), and the one operation with no fallback (thread resolution). --- .claude/knowledge/github-access-paths.md | 66 +++++++++++++++++++ .claude/plans/epoch-recheck-phase0-v1.md | 8 +++ .../mask-membrane-valhalla-integration-v1.md | 7 +- 3 files changed, 80 insertions(+), 1 deletion(-) create mode 100644 .claude/knowledge/github-access-paths.md diff --git a/.claude/knowledge/github-access-paths.md b/.claude/knowledge/github-access-paths.md new file mode 100644 index 0000000..7bfa7fe --- /dev/null +++ b/.claude/knowledge/github-access-paths.md @@ -0,0 +1,66 @@ +# GitHub access paths — three credentials, one rate-limit identity trap + +> READ BY: any session doing PR babysitting, review-thread resolution, +> or hitting a GitHub 403/RATE_LIMIT in this environment. +> Measured 2026-08-28 (PR #47/#48 review cycle), every row exercised +> live — none of this is inferred. + +## The core fact most sessions get wrong + +There are THREE ways out of this container to GitHub, and they carry +only TWO identities: + +| path | credential | identity | +|---|---|---| +| MCP `mcp__github__*` tools | the GitHub App session credential | **user ID 200276742** | +| direct `api.github.com` with `GH_TOKEN` (proxy-bypassed: `curl --noproxy '*'` / Python `ProxyHandler({})`) | the env token (`ghp_…`, strip the wrapping quotes) | **the SAME user ID 200276742** | +| through the session proxy (default env, NO Authorization header you author) | the proxy attaches its OWN credential | **a different identity** | + +Consequence: when the account hits a rate limit, **switching from MCP to +pygithub/curl-with-GH_TOKEN changes nothing** — same identity, same +limit. The tesseract-rs "a 403 is usually the proxy" lesson is about +*authorization* 403s; it does NOT transfer to *rate* limits. The only +path with a different budget is the proxy's own credential. + +## What each path can actually do (measured) + +| operation | MCP | direct GH_TOKEN | session proxy | +|---|---|---|---| +| REST reads (PRs, comments) | ✅ (until limited) | ✅ (until limited) | ✅ `200` | +| REST writes (review-comment **replies**: `POST /pulls/{n}/comments/{id}/replies`) | ✅ | ✅ | ✅ **works — posted live** | +| GraphQL, arbitrary queries | ✅ | ✅ | ❌ "only the pinned set of PR-review operations is served" | +| GraphQL `resolveReviewThread` mutation | ✅ (via `resolve_review_thread`) | ✅ | ❌ rejected — NOT in the pinned set | +| git fetch/push | n/a | ✅ (one-shot token URL rules apply) | ✅ (plain `git push origin`; proxy carries auth) | + +So **review-thread resolution is the one operation with no fallback**: +it is GraphQL-only, the proxy's pinned set excludes the mutation, and +both non-proxy paths share the limited identity. When the account is +limited, the only correct move is to wait for the reset and re-arm a +check-in — do not burn turns cycling transports. + +## Recognizing a SECONDARY (abuse) limit + +Signature, observed live: `GET /rate_limit` reports **5000/5000 +remaining** while GraphQL simultaneously returns +`RATE_LIMIT: API rate limit already exceeded for user ID …`. The quota +endpoint reflects only the primary budget; secondary limits are +invisible there. The `reset` timestamp the quota endpoint names is +still the best available estimate of the cool-off (observed: the limit +held until roughly that time). Do not retry in a loop — secondary +limits extend under hammering. + +## Proxy-path mechanics that save a session ten minutes + +- Plain `curl https://api.github.com/...` with the default env goes + THROUGH the proxy and gets the proxy credential automatically — do + not add an `Authorization` header; authoring one replaces the good + credential with the limited one. +- The proxy's GraphQL rejection message suggests `gh api` — `gh` is not + installed here; plain REST via curl/urllib is the equivalent. +- Replies posted via the proxy land as `claude[bot]`. CodeRabbit + auto-skips engaging with other bots' replies ("Skipped: comment is + from another GitHub bot") — expected, not a failure. +- Token hygiene rules (never in URLs, never printed, strip the wrapping + quotes inline) are unchanged and live in the sibling repos' + CLAUDE.md files (MedCare-rs "GH_TOKEN" section is the canonical + statement); this doc adds the identity/limit map, not new hygiene. diff --git a/.claude/plans/epoch-recheck-phase0-v1.md b/.claude/plans/epoch-recheck-phase0-v1.md index 923180b..e93f21a 100644 --- a/.claude/plans/epoch-recheck-phase0-v1.md +++ b/.claude/plans/epoch-recheck-phase0-v1.md @@ -130,6 +130,14 @@ restoration in `CLAUDE.md` never happens under (b). re-check cannot be brought under 2× the accessor's measured baseline cost — including via the epoch-only export — resolution (b)'s track opens; below 2×, (a) ships without further debate. +- **The measurement runs under the CHOSEN atomicity arm (§3), not the + bare epoch compare.** Arm (i) puts a lock/lease on every access — + a categorically different cost than a compare — so measuring the + bare compare and then shipping (i) would invalidate this gate: the + gate would no longer observe the cost it names, the exact + gate-observability failure the integration plan's standing question + exists to catch. Order is therefore: atomicity decision FIRST, then + the measurement, of that arm. - `cargo test -p lgj-abi` + full Java suite green; clippy `-D warnings`. - Board: `ISS-LGJ-EPOCH-UNCHECKED` flips OPEN → RESOLVED under (a), or OPEN → DOWNGRADED-DOCUMENTED under (b), same commit as the code. diff --git a/.claude/plans/mask-membrane-valhalla-integration-v1.md b/.claude/plans/mask-membrane-valhalla-integration-v1.md index 55cee37..9514554 100644 --- a/.claude/plans/mask-membrane-valhalla-integration-v1.md +++ b/.claude/plans/mask-membrane-valhalla-integration-v1.md @@ -289,7 +289,12 @@ consumer-side scheduling creep F4 forbids. is ns/accessor-call over ≥1M calls on the cached-descriptor accessors, median of 5; the flip fires iff the epoch re-check cannot be brought under **2× the accessor's measured baseline** even via the epoch-only - export. In that case W1.1 pauses on its (a)/(b) decision while W2.1 + export. The measured quantity is the re-check UNDER THE CHOSEN + atomicity arm (the council spec's §3) — a lock/lease per access is a + different cost than a bare compare, so the atomicity decision precedes + the measurement, and an arm-(i) choice legitimately raises the flip's + probability; the flip firing under (i) is a valid outcome, not a + failure of the plan. In that case W1.1 pauses on its (a)/(b) decision while W2.1 proceeds — for its OWN paths only: W2's fused pass and sweeps run through native operations, not the cached-descriptor accessors, so they may be investigated and shipped regardless; nothing that READS