Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions .claude/board/EPIPHANIES.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,91 @@
> `**Status:**`/`**Confidence:**` line. A correction gets its own new,
> dated entry that references the one it corrects — the storno rule.

## 2026-08-27 — E-ZERO-COPY-MEMORY-SAFETY-AUDITED-CLEAN-1

**Status:** AUDITED CLEAN — pinned as normative doctrine in root
`CLAUDE.md` (new "Zero-copy + memory safety" section), same commit.
**Confidence:** High — every item checked against source, not assumed.

Operator issued a 32-point normative zero-copy/memory-safety addendum
(merge-gating). Ran it mechanically against the tree rather than
redesigning from taste, per the session's standing method:

- Provenance-bound generation-checked handles: `registry.rs` —
`fabricated_handles_are_rejected_not_dereferenced`,
`a_reused_slot_invalidates_the_old_handle` (already present).
- Checked-overflow bounds arithmetic: `checked_mul`/`checked_add`
throughout `rowstore.rs`/`kernels.rs` (already present).
- Alignment/endianness as explicit contract fields (never inferred):
`LgjAbiManifest.align_of_*` filled from `core::mem::align_of` on the
real types; `Abi.java` rejects non-little endianness before any
projection (already present).
- Manifest-first handshake ordering (magic → major → minor → sizes →
endianness, `requireMinor(N)` gating): `Abi.java` (already present).
- FFM quarantine: zero `java.lang.foreign`/`java.lang.invoke` in any
public signature; internal use in `RowStore.java`/`FacetMatchView.java`
is private-field-only (verified by grep for `public .*MemorySegment/
Arena/ValueLayout/MethodHandle` — zero hits).
- Named/bounded materialization only: every production `long[]`/
`copyOf`/`toArray` traces to `Mask.materializeRows()`, the manifest
handshake's own name-string read, or `rowLayoutProbe`'s ≤32-byte
diagnostic — no hidden proportional-to-n_rows copy.
- Independently-derived layout parity (not self-compared):
`AbiContractTest`, with a deliberately-impossible-expectation arm
proving the check can fail.
- SIMD backend diagnostic-only: `NativeRuntime.simdBackend()` is a
manifest string; zero `if` branches on it in `src/main`.
- Worker topology substrate-private: zero `workers(`/`workerCount`/
`parallelism(`/`threads(` in production Java/ABI/exports (reconfirmed
from the prior turn's audit, unchanged).

**Verdict: no gap found, nothing redesigned.** The addendum's own
mandatory checklist (§32) is answered YES on every line by structure
already in the tree — this entry and the CLAUDE.md section exist to
PIN the doctrine as merge-gating going forward, not to fix a defect
found today. Any future PR touching the membrane is reviewed against
this section directly.

## 2026-08-27 — E-EXP-KIA-A2-64K-CONVERGENCE-TAIL-DOMINATES-1

**Status:** MEASURED — in-tree, reproducible, this run banked verbatim at
`.claude/board/exp-kia-a2-64k-fresh-run.txt`.
**Confidence:** High for this host/harness; explicitly NOT the operator's
out-of-tree 125ms/233ms weather prior (different workload, per
`mask-native-navigation-correction-v1.md` §8.3 — that gap stays open).

Fresh `lance-graph-supervisor/examples/measure_wal_curve` release run
(lance-graph@1d7bc1b1), answering "measure the 64k execution end first"
before any `BatchWriter`/kanban seam design: the EXP-KIA-A2-64K arm's
**compute** phase parallelizes as documented (21.8ms seq → 6.68ms @
workers=8, 3.27x, matching the plan's own "~3.2-3.5x on 4 cores" prior
exactly) — but compute is the SMALL part of a cycle. Per-cycle steady-state
total (build/population excluded, sequential vs best-parallel):

```
workers=1: think 21.81 + cast 21.52 + collect 10.06 + wal 14.62 + apply 17.73 = 85.74 ms
workers=8: think 6.68 + cast 20.83 + collect 9.45 + wal 15.62 + apply 15.87 = 68.44 ms
```

**The cast/collect/wal/apply tail is FLAT across every worker count
(1/2/4/8/16) — it does not shrink when compute parallelizes** — and it is
~64-65ms of every cycle regardless: ~90% of the workers=8 total, ~10%
compute. Sequential-vs-parallel sealed-cycle digests MATCH at every worker
count (correctness holds; this is a WHERE-does-the-time-go finding, not a
correctness one).

Consequence for the seam question root `CLAUDE.md` names ("64K COMPUTE WAS
PARALLEL … not yet the production loop"): parallelizing compute alone
recovers at most ~15ms of an ~85ms cycle in this harness. The convergence
boundary this repo's own maxims already flag as sequential-by-design
(ONE COMPUTATION IS NOT ONE LANCE WRITE; the SOLE native writer) is where
the time actually is — any seam design that only speeds up compute is
optimizing the part that was never the bottleneck on this measurement.
Cross-ref `CLAUDE.md`'s compute-model maxims and the GridLake block
(deterministic-landing-identity gate) — this measurement is evidence FOR
prioritizing that gate's resolution over a parallel-compute seam, not
against parallel compute itself.

## 2026-08-27 — E-JAVA-IS-SIMD-RS-VALHALLA-PANAMA-IS-THE-POLYFILL-1

**Status:** DOCTRINE — [OPERATOR-FRAMED]. Pinned as the ENFORCEMENT LAYER in
Expand Down
62 changes: 62 additions & 0 deletions .claude/board/PR_ARC_INVENTORY.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,68 @@
> anti-pattern the imported board rules name. Backfilled below in one
> pass rather than left stale; PR #4 onward gets its entry at merge time.

## PR #44 — R1→minor 10: mask algebra restored, the reduction repatriated, the doctrine pinned, the columnar store landed (merged 2026-08-27, merge — 5 commits, head `bd6f666`)

- **Added, as one arc** (the PR body predates its own last three commits —
it says "R2 measured, not landed", which commit 5 then landed; this entry
is the current record):
1. **R1** (`0385269`) — `lgj_hop` selects with `src ∧ class_f ∧ struct_f`,
word-parallel; the walk only EMITS. F2 closed: the structured-edge gate
was an `if` in EVERY prior version incl. #22's — it is the same strided
primitive at `+12`, one call site, zero new kernels.
`facet_bits`/`facet_cache` deleted (a stored projection). Byte-identical
(pinned 10/19/29), and honestly a 19× regression on AoS — the layout
named as the defect, per R11's prior 9.2× pricing.
2. **§13 de-staled** (`ba377b3`) — abi.md described the pre-R1
composition; the #39-shaped prose-lag caught by adversarial re-read.
3. **Minor 9** (`c3ecf37`) — `lgj_rowstore_facet_match_count`: the
facet-match reduction computed where the data is, after THREE Java-side
shapes of it (segment popcount loop; 32 composed counts summed in
Java; a proposed buffer-popcount symbol). Two independent oracles;
both compat directions vs a real minor-8 library. Exposed en route:
root-invoked release builds were silently REFUSED (toolchain floor,
error hidden by tail-piping) — earlier R1 Java runs had loaded a
pre-R1 `.so`; harmless (no observable change) and caught by the
minor-9 gate itself.
4. **Doctrine** (`824996d`) — the simd.rs isomorphism as root CLAUDE.md's
ENFORCEMENT LAYER E1–E6 (Java=facade/37-fns-0-instructions,
Valhalla+Panama=polyfill, Rust=backends/488-intrinsics; scalar is a
backend BELOW the facade; facade intrinsics only as cfg(test) oracles).
J2 closed: `Layouts` derives the geometry, the facade names it.
CODEX_REVIEW_CHECKLIST §8 added (the "saves a crossing" tell).
5. **Minor 10** (`bd6f666`) — `lgj_rowstore_open_columnar`: facet-major
over the (row × facet) plane, same 512n bytes/draws/content (pinned,
bytes-differ anti-vacuity). Lane table 33→97 (lo64/hi32 lanes join
classid) so EVERY field is descriptor-served; Java proven LAYOUT-BLIND
(accessors read only via descriptors; `rowOffset` + last facade
geometry constants deleted; disable-run red at row 1 facet 0 —
the first address divergence — AoS green). Register-sweep family
refuses facet-major with new `UNSUPPORTED_LAYOUT` (−18), two-sided.
- **Locked:** HOP EXECUTES AS MASK × CLASSVIEW → MASK, at every layer with
a named gate; a layout is a SCHEMA (constructor + descriptors, never a
resource kind); carving groups ≤ 4 B and 64-alignment of 512/regions/
blocks pinned as tests (`carving_groups_fit_the_flattening_budget…`) —
the substrate half of R4/R10's Valhalla measurements.
- **Measured, banked on the board:** AoS mask algebra 19× WORSE than the
old sweep (`hop-mask-algebra-vs-columnar.txt` — the finding that forced
minor 10); through the REAL ABI, columnar hop **4.7×/5.9×/3.8×** over AoS
at classid/2-hop/full arms, equivalence asserted before timing
(`columnar-store-abi-bench.txt`).
- **Deferred, named:** the fused single-plane pass (~10× further, per the
lab arm); the register-sweep family on facet-major (an honest −18, not a
gap); Vector API permanently lab (E4).
- **Docs:** abi.md §13 rewrite, §18 new, symbol count 25→26, status −18,
minor 9+10 history; root CLAUDE.md E1–E6.
- **Gates:** Rust 138/139 (both feature configs) · clippy `-D warnings` +
fmt · Java **314 core** (ColumnarStoreTest 10 new) **+ 143 consumer** ·
runtime-confirmed `abi 0.10` · OldAbiCompatTest both directions vs REAL
minor-8 AND minor-9 libraries built from prior commits in worktrees
(path deps resolve relative to the worktree — it must sit beside the
sibling repos, not in /tmp).
- **Confidence:** high — every claim above is a pinned test, a banked
measurement, or a disable-run observed red-then-green; the one narrative
caveat is that the PR BODY describes only commit 1's state.

## PR #42 — lgj-abi: the REAL OGAR ClassView provider, bound behind a feature (merged 2026-08-27, `507cc93`)

- **Added:** `ogar-classview` feature on `native/lgj-abi` binding
Expand Down
Loading