Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 64 additions & 3 deletions docs/abi.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ cannot disagree with itself.
The ABI is a **machine membrane**. It is not the product. The product is the Java
semantic API (see `architecture.md`). Therefore:

- It is **small** — currently 23 symbols (minor 6; the "14" this line carried
- It is **small** — currently 24 symbols (minor 7; the "14" this line carried
at minor 1 was arithmetic drift — the §7 list it referred to already
enumerated 15). Growth is a design smell to be argued for, not a default;
minor 2's three additions are argued in §11, minor 3's one addition in
Expand All @@ -79,7 +79,7 @@ semantic API (see `architecture.md`). Therefore:

```
LGJ_ABI_MAJOR = 0 // incompatible change ⇒ bump; Java refuses to load
LGJ_ABI_MINOR = 6 // additive change ⇒ bump; older Java may still load
LGJ_ABI_MINOR = 7 // additive change ⇒ bump; older Java may still load
LGJ_MAGIC = 0x4C_47_4A_5F_41_42_49_00 // "LGJ_ABI\0" big-endian-read
```

Expand Down Expand Up @@ -132,6 +132,8 @@ required — a gate that rejected everything would satisfy a rejection-only test
- **Minor 4** (2026-08-18, D-LGJ-W8) — `lgj_mask_andnot` (mask complement)
and `lgj_hop` (one-hop graph traversal, gated by the
`lance-graph-contract` `ClassView`/`FieldMask` LAW — §13).
- **Minor 7** (2026-08-25) — `lgj_row_layout_probe` (§16): the whole-row
alignment answer, all 32 facets in one crossing. No new status.
- **Minor 6** (2026-08-25) — `lgj_reduce_facet_sum_resolved` (§15): the same
sweep, but under the grouping the POPULATION resolves to via
`ClassView::cascade_shape`, rather than one the caller asserts. One new
Expand Down Expand Up @@ -326,7 +328,7 @@ predicates or rows are involved. The unfused per-predicate ops are retained only
so the fused path can be benchmarked *against* something and so parity can be
checked predicate-by-predicate.

## 7. The function surface (23 symbols)
## 7. The function surface (24 symbols)

All symbols are prefixed `lgj_`. All return `i32` status except the manifest
getter. `out_*` parameters are written only on `OK`.
Expand Down Expand Up @@ -1006,3 +1008,62 @@ answer rather than reading as an empty memo.
A test-only counter (`RESOLUTIONS`) makes the memo's behaviour observable rather
than asserted: the first sweep resolves, five repeats over the same population do
not, a different facet does, and a rewritten population does.

## 16. The whole-row layout probe (ABI minor ≥ 7)

```
i32 lgj_row_layout_probe(u64 res, u64 mask, u8* out, u64 out_len)
```

For **every** facet, the SET of register groupings its selected rows carry. One
crossing covers all 32 — asking per facet would be 32 crossings, and is how a
consumer drifts into the per-element loop §6 forbids.

### Alignment as arithmetic, not a scan

Each output byte is a 3-bit set (bit `w` = some row resolves to grouping `w`)
plus bit 3 for "some row's classid has no `ClassView` answer". Then:

```
aligned(facet) ⟺ popcount(byte) == 1 && byte & UNANSWERABLE == 0
```

One `or` per (row, facet), no comparison and no early exit, so cost does not
depend on the data. **An OR-accumulated set is exact where cheaper accumulators
are not:** a sum of wire values cannot tell `{0,2}` from `{1,1}`, and an XOR
cannot tell `{1,1}` from `{}`. The set forgets multiplicity, which is precisely
the information the question does not need.

`0` means the EMPTY set — no row selected — and is deliberately distinguishable
from disagreement. Conflating them would report "misaligned" for a population
that simply is not there.

### What it measured, immediately

A mask from `lgj_op_eq_classid(facet 3, …)` constrains **facet 3 only**; the
other 31 facets of those rows carry whatever classids the generator gave them.
Measured on the fixture: **1 of 32 facets aligned.** A test asserting
`isFullyAligned()` there failed, and the expectation was wrong rather than the
code — which is exactly the confusion a whole-row probe exists to remove.

Note what this says about **placement**: the canon makes `classid` the key's
prefix precisely so key-ordered placement clusters a class into a range. The
fixture generates classids uniformly at random instead — measured mean run
length **1.07**, mean gap ~17 rows. Clustering is worth doing, and measured at
this stride it is worth ~2× (12.6 vs 25.6 ns/row for the same population size,
contiguous vs every-16th-row) — not from cache-line count, which is one line per
row either way at a 512-byte stride, but from stride-predictable prefetch and TLB
locality.

### A classid is a GLOBAL address

The `classid → grouping` table is process-global (`LazyLock`, 64 KiB, built
once), not per dataset: the same classid means the same class in every SoA, so
the resolution is dataset-independent. An earlier version put it on `RowStore`,
which was wrong in shape rather than output — the answers were right, but the
placement implied two datasets could disagree about what a classid carves into,
which the address space does not permit.

And the table captures **layout only**. Meaning, RBAC, ontology category and
render template are separate resolutions off the same address; none belong in it
and none can be inferred from it.
4 changes: 4 additions & 0 deletions java/src/main/java/com/adaworldapi/lancegraph/FacetId.java
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,10 @@
*/
public record FacetId(int index) {

/** Facets per row — 32, the canonical `32 × 16 B = 512 B` row (abi.md §11). */
public static final int COUNT = 32;


public FacetId {
if (index < 0 || index > 31) {
throw new IllegalArgumentException("facet index must be in 0..31, was " + index);
Expand Down
98 changes: 98 additions & 0 deletions java/src/main/java/com/adaworldapi/lancegraph/RowLayout.java
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
package com.adaworldapi.lancegraph;

import java.util.Optional;

/**
* Which register grouping each of a row store's facets carries, for one selection — the whole-row
* alignment answer (abi.md §16).
*
* <p><strong>Alignment is arithmetic here, not a scan.</strong> The native side accumulates, per
* facet, a 3-bit SET of the groupings its selected rows resolve to (plus one bit for "some row's
* classid has no ClassView answer"). A facet is aligned exactly when that set has a single member:
*
* <pre>
* aligned(facet) ⟺ bitCount(set) == 1 &amp;&amp; no unanswerable bit
* </pre>
*
* <p>An OR-accumulated set is exact where cheaper accumulators are not: a sum of wire values cannot
* tell {@code {0,2}} from {@code {1,1}}, and an XOR cannot tell {@code {1,1}} from {@code {}}. The
* set forgets multiplicity, which is exactly the information the question does not need.
*
* <p>One crossing covers all 32 facets. Asking per facet would be 32 crossings and is how a
* consumer drifts into the per-element loop abi.md §6 forbids.
*/
public final class RowLayout {

/** Bit meaning "some selected row's classid had no ClassView answer". */
private static final int UNANSWERABLE = 0b1000;

private final byte[] sets;

RowLayout(byte[] sets) {
this.sets = sets;
}

/** How many facets this covers. */
public int facetCount() {
return sets.length;
}

/**
* The grouping facet {@code f}'s selected rows all share, or empty when they do not.
*
* <p>Empty covers three genuinely different situations, and a caller that needs to tell them
* apart should use {@link #isAligned}, {@link #isEmpty} and {@link #hasUnanswerable}: the rows
* disagree; some row's classid is unanswerable; or no row was selected at all.
*/
public Optional<Carving> carvingOf(FacetId f) {
int set = sets[f.index()] & 0xFF;
if (Integer.bitCount(set) != 1 || (set & UNANSWERABLE) != 0) {
return Optional.empty();
}
return Optional.of(Carving.ofWire(Integer.numberOfTrailingZeros(set)));
}

/** Whether facet {@code f}'s selected rows all read the register the same way. */
public boolean isAligned(FacetId f) {
int set = sets[f.index()] & 0xFF;
return Integer.bitCount(set) == 1 && (set & UNANSWERABLE) == 0;
}

/** Whether no selected row carried this facet — the empty set, distinct from disagreement. */
public boolean isEmpty(FacetId f) {
return sets[f.index()] == 0;
}

/** Whether some selected row's classid had no ClassView answer at this facet. */
public boolean hasUnanswerable(FacetId f) {
return (sets[f.index()] & UNANSWERABLE) != 0;
}

/** Whether EVERY facet is aligned — the whole row reads uniformly. */
public boolean isFullyAligned() {
for (int i = 0; i < sets.length; i++) {
int set = sets[i] & 0xFF;
if (Integer.bitCount(set) != 1 || (set & UNANSWERABLE) != 0) {
return false;
}
}
return true;
}

/** How many facets are aligned. */
public int alignedCount() {
int n = 0;
for (int i = 0; i < sets.length; i++) {
int set = sets[i] & 0xFF;
if (Integer.bitCount(set) == 1 && (set & UNANSWERABLE) == 0) {
n++;
}
}
return n;
}

@Override
public String toString() {
return "RowLayout[" + alignedCount() + "/" + sets.length + " facets aligned]";
}
}
17 changes: 17 additions & 0 deletions java/src/main/java/com/adaworldapi/lancegraph/RowStore.java
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,23 @@ public Mask maskOfFacetClass(FacetId facet, int classId) {
return new Mask(this, mask);
}

/**
* For every facet, which register grouping this selection's rows carry (abi.md §16) — the
* whole-row alignment answer, in ONE crossing.
*
* <p>Use this to ask "is this population layout-aligned?" before sweeping it, or to discover
* which facets a heterogeneous selection can still be swept on. Asking {@link #facetSum} per
* facet to find out would be 32 crossings and would fail on the misaligned ones rather than
* reporting them.
*
* @param selection the rows to probe; must belong to this store
*/
public RowLayout layout(Mask selection) {
java.util.Objects.requireNonNull(selection, "selection");
requireOpen("layout()");
return new RowLayout(Engine.rowLayoutProbe(handle, selection.handle(), FacetId.COUNT));
}

/**
* Sum one facet's 12-byte register under the grouping the SELECTION ITSELF resolves to
* (abi.md §15, ABI minor 6) — the verified sibling of {@link #facetSumAs}.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -439,6 +439,15 @@ private static final class Minor6 {
private Minor6() {}
}

/** The whole-row layout probe (docs/abi.md §16, ABI minor 7). */
private static final class Minor7 {
static final MethodHandle ROW_LAYOUT_PROBE = mh("lgj_row_layout_probe",
FunctionDescriptor.of(ValueLayout.JAVA_INT, ValueLayout.JAVA_LONG,
ValueLayout.JAVA_LONG, ValueLayout.ADDRESS, ValueLayout.JAVA_LONG));

private Minor7() {}
}

/**
* Sum one facet's 12-byte register, under {@code carving}, over the rows a mask selects.
*
Expand Down Expand Up @@ -476,6 +485,21 @@ public static long reduceFacetSumResolved(long res, int facet, long mask, Memory
return outSum.get(ValueLayout.JAVA_LONG, 0);
}

/**
* For every facet, the SET of register groupings the selected rows carry — one crossing for
* all 32 facets.
*/
public static void rowLayoutProbe(long res, long mask, MemorySegment out, long outLen) {
crossed();
int st;
try {
st = (int) Minor7.ROW_LAYOUT_PROBE.invokeExact(res, mask, out, outLen);
} catch (Throwable t) {
throw wrap("lgj_row_layout_probe", t);
}
Status.check("lgj_row_layout_probe", st);
}

// ── row store (docs/abi.md §11, ABI minor 2) ─────────────────────────────────────────────
//
// Callers above this class are expected to have already checked Abi.requireMinor(2) — these
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -210,6 +210,19 @@ public static long openRowStoreWithEdges(long nRows, long seed, int edgeClassid,
* Sum one facet's register under the grouping the POPULATION resolves to (abi.md §15, ABI
* minor 6), returning {@code [sum, carvingWire]}. Requires ABI minor &gt;= 6.
*/
/**
* For every facet, the set of groupings its selected rows carry (abi.md §16, ABI minor 7).
* Requires ABI minor &gt;= 7.
*/
public static byte[] rowLayoutProbe(long store, long mask, int facets) {
Abi.requireMinor(7);
try (Arena arena = Arena.ofConfined()) {
MemorySegment out = arena.allocate(facets);
Downcalls.rowLayoutProbe(store, mask, out, facets);
return out.toArray(ValueLayout.JAVA_BYTE);
}
}

public static long[] facetSumResolved(long store, int facet, long mask) {
Abi.requireMinor(6);
Scratch s = SCRATCH.get();
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,64 @@ public static void run(Checks c) {
}
}

c.section("the whole-row layout probe (abi.md §16): alignment answered by arithmetic,"
+ " for all 32 facets in ONE crossing");
try (RowStore store = RowStore.open(1000, SEED)) {
// A single-class selection: every facet's rows share a classid, so every facet must
// be aligned, and facet 3's answer must match what facetSum resolves.
//
// NOTE, and it is the whole point of having this probe: a mask built by
// maskOfFacetClass(facet 3, classid 3) constrains FACET 3 only. The other 31 facets
// of those same rows carry whatever classids the generator gave them, so the ROW is
// not aligned even though the facet is. An earlier version of this test asserted
// isFullyAligned() here and failed — the expectation was wrong, not the code, and
// that is exactly the confusion a whole-row probe exists to remove.
try (Mask single = store.maskOfFacetClass(FacetId.of(3), 3)) {
RowLayout l = store.layout(single);
c.eq("all 32 facets covered", 32L, (long) l.facetCount());
c.that("the facet the mask was BUILT on is aligned", l.isAligned(FacetId.of(3)));
c.eq("facet 3's probed grouping equals what facetSum resolves",
store.facetSum(FacetId.of(3), single).carving(),
l.carvingOf(FacetId.of(3)).orElseThrow());
c.that("but the ROW is not aligned — the other facets are unconstrained",
!l.isFullyAligned());
c.note("aligned facets on a facet-3 single-class selection: "
+ l.alignedCount() + "/32");
}

// A mixed-grouping selection: facet 3 must report NOT aligned, and facetSum must
// refuse it. The two surfaces must agree about the same population.
try (Mask a = store.maskOfFacetClass(FacetId.of(3), 3);
Mask b = store.maskOfFacetClass(FacetId.of(3), 4)) {
long[] ra = a.materializeRows();
long[] rb = b.materializeRows();
long[] mixed = new long[ra.length + rb.length];
System.arraycopy(ra, 0, mixed, 0, ra.length);
System.arraycopy(rb, 0, mixed, ra.length, rb.length);
try (Mask both = store.importRows(mixed)) {
RowLayout l = store.layout(both);
c.that("facet 3 is NOT aligned on a mixed-grouping selection",
!l.isAligned(FacetId.of(3)));
c.that("and its grouping is therefore absent",
l.carvingOf(FacetId.of(3)).isEmpty());
c.that("which is not the same as empty", !l.isEmpty(FacetId.of(3)));
c.throwsUp("facetSum refuses the same population", RuntimeException.class,
() -> store.facetSum(FacetId.of(3), both));
}
}

// An empty selection: every facet reports the EMPTY set, which must be
// distinguishable from disagreement — otherwise "not aligned" would conflate
// "no rows" with "rows that disagree".
try (Mask empty = store.importRows()) {
RowLayout l = store.layout(empty);
c.that("an empty selection reports empty, not misaligned",
l.isEmpty(FacetId.of(0)) && l.isEmpty(FacetId.of(31)));
c.that("empty is not aligned either", !l.isAligned(FacetId.of(0)));
c.eq("so nothing counts as aligned", 0L, (long) l.alignedCount());
}
}

c.section("Carving's own invariant: every reading covers exactly the 12-byte register");
for (Carving carving : Carving.values()) {
c.eq(carving + " covers 12 bytes", 12L,
Expand Down
2 changes: 1 addition & 1 deletion native/lgj-abi/src/abi.rs
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ pub const LGJ_ABI_MAJOR: u32 = 0;
///
/// `docs/abi.md` §13). Purely additive: a minor-3 Java loads fine and
/// simply cannot call either new symbol.
pub const LGJ_ABI_MINOR: u32 = 6;
pub const LGJ_ABI_MINOR: u32 = 7;

/// `"LGJ_ABI\0"` read big-endian.
///
Expand Down
Loading