R1 — the hop's selection is mask algebra again (and R2 is measured, not landed) - #44
Merged
Conversation
Operator ruling: there is no gathering. Walking src's set bits is a serialization of a population that is already there, whether or not it allocates — avoiding the Vec while doing it with a loop counter is the same act in the time dimension. lgj_hop now selects with src AND class_f AND struct_f, word-parallel. No row is examined to decide whether it participates; the walk only EMITS from the result, and only because the destination index is decoded from the selected row's payload (the operand of a permutation, not a decision about membership). F2, which no PR in this arc had caught: the structured-edge gate (payload_hi32 != 0) was an `if` inside the row walk in EVERY version, including PR #22's clean one. It is a per-row equality against zero — the same strided primitive as the classid match, twelve bytes further into the facet. simd_rowstore_u32_eq_mask takes an arbitrary first_offset, so both predicates are one call each and the gate cost one call site, not a kernel. facet_bits / facet_cache / FACET_CACHE_SLOTS deleted. Under the format-string reading of the 4+12 facet the memo cached the interpolated string; the projection is applied at read, never stored. Byte-identical: 134/134 Rust including the pinned 10/19/29 regression, and 447/447 Java unchanged (304 core + 143 consumer). AND IT IS A 19x REGRESSION AS IT STANDS, which is the finding rather than a side note. 65 536 rows: sweep 2 126 us -> 40 632 us, flat in density. 32 facets x 2 predicates is 64 whole-population passes at stride 512, ~2 GB of traffic to read 512 KB. R11 priced this layout at 9.2x before the arc began; PR #40 diagnosed it as an algebra defect and banked the opposite as a law. R2 measured as a lab arm (R11 precedent, no ABI change): the canvas is the (row x facet) plane, not the row. Same bytes field-major -> one contiguous pass per predicate, a PERIODIC participation operand (64 slots per word = 2 rows x 32 facets), src expanded by splat. 902-2 271 us: ~40x the AoS mask shape, 2.3-5.7x the sweep, beating the gather outright at full density, with cost tracking the canvas rather than the frontier. Equivalence asserted at every configuration; raw output banked on the board. The columnar STORE is not landed. Until it is, the hop is lawful and slow, and that trade is deliberate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_2b106312-e236-4fdc-b4e4-96b9ea7abeda) |
The "Kernel composition" paragraph still described the single-predicate shape: one classid match per facet, with the structured-edge gate absent because it was an `if` in the walk rather than an operand. R1 makes selection `src AND class_f AND struct_f`, so §13 now names both predicates, the one primitive that produces both (same call, offsets +0 and +12 into the facet), and the reason the scatter alone stays scalar — the destination is decoded from the selected row's payload, which makes it the operand of a permutation rather than a decision about membership. Caught by re-reading the diff adversarially rather than by a gate. Leaving prose behind a changed function is exactly what #39 did and #40 had to fix one PR later; no reason to make the same reviewer spend that cycle twice. No ABI change: no symbol, no signature, no minor bump. Gates re-run unchanged — 134/134, clippy -D warnings, fmt. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
Operator ruling: Java hands decorative where() through Panama; Rust does mask ops, only — and Java doesn't even know mask count. FacetMatchView.cardinality() violated this in three successive shapes, each one layer up from the last: a Java popcount loop over a fetched segment (doc-commented "deliberately Java-side" to save a crossing); then 32 composed per-facet mask counts summed in Java — every operation native, but the decomposition still executing in Java, which is still Java holding a moving part; and the first proposed fix, "add a popcount symbol over the buffer", which asked how to reduce a buffer Java should never hold. Minor 9, one symbol: lgj_rowstore_facet_match_count. Sigma over facets of popcount(class_f), computed natively with the same strided-equality mask the classid ops use plus the sanctioned ndarray::simd popcount. One crossing in, one u64 back; Java neither iterates facets nor sums partials and does not learn that the answer has parts. cardinality() is a single delegation through a package-private RowStore bridge. Falsifier: the count against TWO independent oracles (the lgj_row_facet_match buffer popcount — the very reduction Java used to do — and a scalar recompute sharing no kernel), absent-needle zero, null-out rejection. Both gate directions proven against a REAL minor-8 library built from main in a worktree: AbiMismatchException naming minor 9, never a bare missing symbol, never a fallback Java loop. OldAbiCompatTest 8/8. The stale-.so iron rule fired on me during this change: the root-invoked release build was silently refused (root resolves the default toolchain, below the 1.97 floor; the error hidden by tail-piping), so earlier R1 Java runs loaded a pre-R1 .so — harmless only because R1 changes no observable behaviour, and surfaced precisely by the minor-9 requireMinor gate. Correct build: inside native/lgj-abi with CARGO_TARGET_DIR at the root target. Gates: Rust 135/135 (both feature configs), clippy -D warnings + fmt clean; Java 304 core + 143 consumer = 447 against the minor-9 .so (abi 0.9 reported at runtime); abi.md symbol count 25 + minor-9 history; board entry same commit. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
… under E3
Operator frame, verified in-tree before pinning: Java is simd.rs (the
facade — 37 functions, zero shipping instructions; raw intrinsics only
under cfg(test) as oracles), Valhalla+Panama is the polyfill (one source,
zero cost where the platform provides it, still correct where it does
not), Rust is simd_{amx,avx512,avx2,neon,wasm,scalar}.rs (all machinery —
488 intrinsics in one backend; the scalar fallback is a BACKEND below the
facade, never inline in it). The stack nests: lgj's bottom is ndarray's
top.
Pinned in root CLAUDE.md as rules E1-E6, each with its named gate: no Java
compute path (E1, G2 + the allowlist); Java scalar only as test oracle
(E2); geometry has one spelling owned by the polyfill (E3); Vector API is
permanently a lab arm (E4); capability lands backend-first (E5 = the STOP
rule restated); consumers import only the facade (E6, ApiSurfaceTest as
this repo's simd-savant). Every violation this session found breaks one of
these at the layer it names — the cardinality three-strikes is E1's
provenance, verbatim.
J2 closed under E3: RowStore's hand-written ROW_BYTES = 512 /
FACET_BYTES = 16 and the literal +4 / +12 payload offsets are gone.
Layouts now DERIVES ROW_BYTES, FACET_BYTES, FACET_PAYLOAD_OFFSET
(byteOffset of the payload element) and FACET_PAYLOAD_HI32_OFFSET (payload
offset + the u64's own byteSize — no literal survives), and the facade
names them. One source, proven by the existing SELF_CHECK; the enforcement
is the deletion of the second spelling, not a tautological test asserting
an expression equals itself.
Board: EPIPHANIES E-JAVA-IS-SIMD-RS-VALHALLA-PANAMA-IS-THE-POLYFILL-1,
LATEST_STATE entry, CODEX_REVIEW_CHECKLIST gains section 8 (the five
greppable review items, with the "saves a crossing" tell named) — same
commit as the code, per the board rule.
Gates: Java 304 core + 143 consumer = 447 unchanged; no Rust change; no
public signature moved.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
R2 stops being a lab arm. lgj_rowstore_open_columnar opens a facet-major
store over the (row x facet) plane — classid / lo64 / hi32 regions, each 32
contiguous per-facet blocks, same 512n bytes, same generator draws, same
logical content (pinned, with an anti-vacuity guard that the BYTES differ).
A layout is a schema, not a resource kind: every mask, hop and count symbol
takes the handle unchanged and answers identically — the 10 -> 19 -> 29 hop
regression is pinned on BOTH layouts, facet-match buffers byte-identical,
eq-classid counts equal per facet.
Measured through the REAL ABI (65 536 rows, all 32 facets, equivalence
asserted before timing; banked at .claude/board/columnar-store-abi-bench.txt):
hop 4.7x / 5.9x / 3.8x over AoS at classid-frontier / 2-hop-frontier /
full-population arms. The lab's fused single-plane pass has a further ~10x
in it — named as the next rung, not smuggled into this one.
The lane table is the mechanism (33 -> 97): payload lo64/hi32 lanes join
classid, every descriptor carrying its own layout's offset and stride. The
generic strided-eq kernel takes the stride from the layout's lane algebra
(RowLayout::{classid,lo64,hi32}_lane — ONE source), so eq-classid, the hop
predicates, facet-match and the native count are all layout-aware with zero
new SIMD.
Java is proven LAYOUT-BLIND: RowStore's accessors read only through served
lane descriptors; rowOffset and the facade's last geometry constants are
deleted (E3 structural on both sides). Disable-run, two-sided: stride
hard-coded to 512 fails the columnar store at row 1 facet 0 — the first row
where the layouts' addresses diverge — while AoS stays green; restore, 10/10.
Honest refusal over silent wrongness: the register-sweep family
(lgj_reduce_facet_sum{,_resolved}, lgj_row_layout_probe) reads the 12-byte
payload as ONE contiguous register, which facet-major deliberately splits —
new status UNSUPPORTED_LAYOUT (-18), pinned two-sided (same calls succeed
on AoS). Re-gathering the register per row would be the serialization this
arc exists to forbid.
The operator's stated platform facts are pinned as tests rather than
trusted: every carving group (6x2 / 4x3 / 3x4) is <= 4 bytes — half the
JEP 401 flattening budget R4/R10 measured from the Valhalla side, so the
group flattens and the register never does — and 512 plus every columnar
region/block offset is 64-aligned for any n.
Gates: Rust 138/139 both feature configs, clippy -D warnings + fmt; Java
314 core (ColumnarStoreTest 10 new) + 143 consumer, runtime-confirmed
abi 0.10; OldAbiCompatTest proven BOTH directions against a real minor-9
library built from the previous commit (the minor-10 gate names the minor,
never a missing symbol). Lane-table growth re-pinned as contrast (lane 34
EXISTS as lo64 now; 97 is the first rejecting id). docs/abi.md sect. 18,
symbol count 26, status table -18, minor history; board entries same commit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DCfrD5y19cvFc4AoyydXYv
AdaWorldAPI
marked this pull request as ready for review
August 27, 2026 20:51
AdaWorldAPI
added a commit
that referenced
this pull request
Aug 28, 2026
board: PR #44 arc entry (R1 → minor 10, the whole arc)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
There is no gathering. Walking
src's set bits is a serialization of a population that is already there — avoiding theVecand doing it with a loop counter is the same act in the time dimension.R1 — what changed
lgj_hopselects withsrc ∧ class_f ∧ struct_f, word-parallel. No row is examined to decide whether it participates. The walk only emits from the result, and only because the destination index is decoded from the selected row's payload — the operand of a permutation, not a decision about membership.F2, which no PR in this arc had caught: the structured-edge gate
payload_hi32 != 0was anifinside the row walk in every version, PR #22's clean one included. It is a per-row equality against zero — the same strided primitive as the classid match, twelve bytes further into the facet.simd_rowstore_u32_eq_masktakes an arbitraryfirst_offset:first_offsetneedlefcarries classEf*16 + 0Efis a structured edgef*16 + 120One call site, zero new kernels.
facet_bits/facet_cache/FACET_CACHE_SLOTSdeleted. Under the format-string reading of the 4+12 facet —classid -F payload, PowerShell"{0} {1}" -F $1, $2— the memo cached the interpolated string. The projection is applied at read, never stored.Byte-identical: 134/134 Rust incl. the pinned 10/19/29 regression (which is the proof the answer didn't move), 447/447 Java unchanged.
…and it is a 19× regression as it stands
Stated up front because it is the finding, not a side note. 65 536 rows:
(µs, median of 7, release, shared container. Equivalence asserted at every configuration — all four shapes byte-identical.)
32 facets × 2 predicates = 64 whole-population passes at stride 512, ~2 GB of traffic to read 512 KB of classids. 1 024 rows → 144 µs; 65 536 rows → 40 632 µs — 282× for 64× the rows, cache and TLB failing together.
The algebra is right; the layout is the defect. R11 (#31) priced it at 9.2× before this arc began. PR #40 diagnosed it as an algebra defect and banked the opposite as a law — a measurement taken inside the defect, generalised as a property of the operation. Superseded here.
R2 — measured as a lab arm, not landed
R11's precedent: measure the layout before changing the store. The correction the measurement forced is that "give classid its own lane" is still 32 strided passes — the canvas is the
(row × facet)plane, not the row. Same 512 bytes, field-major:classandstructare one contiguous pass each, no stride;u64, not a buffer;srcexpands 1 row-bit → 32 slot-bits by splat.Four operands, three ANDs, and not one built by asking which rows were selected. ~40× the AoS mask shape, 2.3–5.7× the sweep, beats the gather outright at full density — and cost tracks the canvas, not the frontier: 2.5× across a 10 000× density range.
Honest boundaries
lgj_hopis lawful and slow, and that trade is deliberate: the currency stays correct and the physical layer is the named blocker.Board:
LATEST_STATEentry,ISS-LGJ-HOP-LAYOUT-BLOCKS-THE-ALGEBRA, raw output at.claude/board/hop-mask-algebra-vs-columnar.txt— all in this commit.Generated by Claude Code