fix(reader): reject duplicate field names and arity desync in file schema

Completes the column-identity work: PostscriptParser rejects a file
schema whose struct declares duplicate field names (VortexException;
the name-keyed Chunk API would silently drop a column) or mismatched
names/dtypes lengths. Deliberate read-side divergence from the Rust
reader's tolerance, documented in docs/compatibility.md: the
reference WRITER refuses such files ("StructLayout must have unique
field names"), so tolerating them buys nothing and silent column
loss is worse than a loud failure.

DType.Struct's canonical constructor now validates non-null lists
and 1:1 arity; duplicate names stay legal in memory, mirroring
Rust's StructFields — uniqueness is a file-boundary contract.
CLAUDE.md file-format note corrected: the dtype blob is FlatBuffers
(FbsDType), not Protobuf.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

Author: dfa1

CLAUDE.md

@@ -115,7 +115,7 @@ git push && git push --tags          # GitHub Actions deploys the tag to Maven C
 
 8-byte trailer at EOF: `version(u16 LE) | postscriptLen(u16 LE) | magic(VTXF)`. The postscript
 (FlatBuffer, immediately before the trailer) points (offset+length) to the Footer (FlatBuffer),
-DType (Protobuf), and Layout (FlatBuffer) blobs elsewhere in the file.
+DType (FlatBuffer), and Layout (FlatBuffer) blobs elsewhere in the file.
 
 Layout tree: `Struct → Zoned(Stats) → Chunked → [Flat, Flat, ...]`
 - **Flat** single encoded segment · **Chunked** sequence of Flats · **Struct** one child/column

TODO.md

@@ -95,23 +95,6 @@ Per-encoding gotchas:
     - See [ADR-0008](adr/0008-domain-primitives-unsigned-integers.md) and https://dfa1.github.io/articles/rethink-domain-primitives-with-valhalla
     - Candidates: `PType` integer kinds, buffer offsets, row indices, byte lengths
     - Goal: type-safety at zero cost (value class = no heap alloc, no boxing)
-- [ ] **Column identity: duplicate/empty field names — remaining reader-side work** — measured
-  against the JNI oracle 2026-07-04 (`ColumnNameEdgeCasesIntegrationTest` pins the facts):
-  - `""` is a legal column name: round-trips both directions, DONE (pinned by tests).
-  - Duplicate names: Rust's in-memory `StructFields` allows them (first-match access) but its
-    FILE writer rejects them ("StructLayout must have unique field names"). Our writer now
-    mirrors that guard, DONE. The Rust READER tolerates a foreign duplicate-name file
-    (schema reports both fields); ours silently collapses them in `Chunk`'s
-    `Map<String, Array>` — a crafted file loses a column with no error. Open: reject loudly at
-    the parse edge (cheap, diverges from Rust reader tolerance) vs ordered first-match `Chunk`
-    columns (Rust-aligned, reshapes the `columns()` API) — ADR when picked up.
-  - `DType.Struct` record constructor validates nothing (arity `fieldNames`/`fieldTypes` desync
-    possible; only `StructBuilder` checks) — consider canonical-constructor validation.
-  - Unexplained: a duplicate-name write produced two chunks from one `writeChunk` (pre-guard);
-    understand before it bites elsewhere.
-  - `ColumnName` domain primitive verdict: per-value invariant is thin (`""` is legal); the real
-    invariant is structural and lives on the struct/file layer, exactly as the reference's error
-    message says.
 
 ## Compute
 

core/src/main/java/io/github/dfa1/vortex/core/model/DType.java

@@ -245,6 +245,25 @@ record Struct(
             java.util.List<DType> fieldTypes,
             boolean nullable
     ) implements DType {
+
+        /// Validates that names and types pair 1:1. Duplicate names are legal in memory,
+        /// mirroring the Rust reference's `StructFields`; uniqueness is a file-boundary
+        /// contract, enforced by the writer and the file parser.
+        ///
+        /// @param fieldNames ordered list of field names
+        /// @param fieldTypes ordered list of field types, parallel to `fieldNames`
+        /// @param nullable   whether null values are permitted
+        /// @throws NullPointerException     if `fieldNames` or `fieldTypes` is `null`
+        /// @throws IllegalArgumentException if the two lists differ in size
+        public Struct {
+            java.util.Objects.requireNonNull(fieldNames, "fieldNames");
+            java.util.Objects.requireNonNull(fieldTypes, "fieldTypes");
+            if (fieldNames.size() != fieldTypes.size()) {
+                throw new IllegalArgumentException("fieldNames/fieldTypes size mismatch: "
+                        + fieldNames.size() + " names, " + fieldTypes.size() + " types");
+            }
+        }
+
         /// Returns the type of the field with the given name.
         ///
         /// @param name the field name to look up

core/src/test/java/io/github/dfa1/vortex/core/model/DTypeStructValidationTest.java

@@ -0,0 +1,50 @@
+package io.github.dfa1.vortex.core.model;
+
+import org.junit.jupiter.api.Test;
+
+import java.util.List;
+
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/// Canonical-constructor validation of [DType.Struct]: names and types must pair 1:1.
+/// Duplicate names stay legal in memory — mirroring the Rust reference's `StructFields` —
+/// because uniqueness is a file-boundary contract (writer guard + parser guard), not a
+/// property of the in-memory type.
+class DTypeStructValidationTest {
+
+    private static final DType I64 = new DType.Primitive(PType.I64, false);
+
+    @Test
+    void construct_arityMismatch_throwsIllegalArgumentException() {
+        // Given / When / Then — two names, one type: the record constructor rejects the desync
+        assertThatThrownBy(() -> new DType.Struct(List.of("a", "b"), List.of(I64), false))
+                .isInstanceOf(IllegalArgumentException.class)
+                .hasMessageContaining("size mismatch");
+    }
+
+    @Test
+    void construct_nullNames_throwsNullPointerException() {
+        // Given / When / Then
+        assertThatThrownBy(() -> new DType.Struct(null, List.of(I64), false))
+                .isInstanceOf(NullPointerException.class);
+    }
+
+    @Test
+    void construct_nullTypes_throwsNullPointerException() {
+        // Given / When / Then
+        assertThatThrownBy(() -> new DType.Struct(List.of("a"), null, false))
+                .isInstanceOf(NullPointerException.class);
+    }
+
+    @Test
+    void construct_duplicateNames_isLegalInMemory() {
+        // Given — duplicate names, mirroring Rust StructFields (uniqueness is enforced at the
+        // file boundary by the writer and parser, not by the in-memory type)
+        // When
+        DType.Struct result = new DType.Struct(List.of("dup", "dup"), List.of(I64, I64), false);
+
+        // Then
+        assertThat(result.fieldNames()).containsExactly("dup", "dup");
+    }
+}

docs/compatibility.md

@@ -35,6 +35,7 @@ resolves only the standalone decoders in `reader`; no encoder class is loaded.
 | `vortex.onpair` experimental string encoding | Rust 0.74.0 | ❌ Not registered. Files using it fail to decode unless `Registry.allowUnknown()` is enabled. |
 | `vortex.variant` arbitrary nested objects | Rust (`vortex.parquet.variant`) | ⚠️ Java encodes/decodes variant columns of **typed scalar** values (constant / chunked-of-constants core, optional shredded child); Java↔Rust round-trip verified. Arbitrary nested JSON objects and real path-based shredding need the `vortex.parquet.variant` physical encoding — deferred ([ADR 0014](../adr/0014-variant-encoding-strategy.md)). |
 | Arrow extension array import affecting Variant shape | Rust 0.74.0 (#8125) | Untested. Re-run integration fixtures against v0.74.0 once published. |
+| Duplicate struct field names | Rust writer rejects ("StructLayout must have unique field names"); Rust reader tolerates foreign files (first-match access) | ⚠️ Deliberate divergence on read: Java rejects such files with `VortexException("duplicate field name in file schema")` instead of tolerating them — the name-keyed `Chunk` API cannot represent both columns, and silent column loss is worse than a loud failure on a file the reference writer refuses to produce. Java's writer mirrors the Rust writer's rejection. Empty column name `""` is legal on both sides (pinned by `ColumnNameEdgeCasesIntegrationTest`). |
 
 ## Encodings
 

reader/src/main/java/io/github/dfa1/vortex/reader/PostscriptParser.java

@@ -20,6 +20,7 @@
 
 import java.lang.foreign.MemorySegment;
 import java.util.ArrayList;
+import java.util.HashSet;
 import java.util.List;
 
 final class PostscriptParser {
@@ -238,10 +239,22 @@ private static DType convertDType(io.github.dfa1.vortex.core.fbs.FbsDType fbs, i
             case FbsType.FbsBinary -> new DType.Binary(fbs.type(new FbsBinary()).nullable());
             case FbsType.FbsStruct_ -> {
                 var s = fbs.type(new FbsStruct_());
+                if (s.namesLength() != s.dtypesLength()) {
+                    throw new VortexException("struct names/dtypes length mismatch: "
+                            + s.namesLength() + " names, " + s.dtypesLength() + " dtypes");
+                }
                 var names = new ArrayList<String>(s.namesLength());
                 var types = new ArrayList<DType>(s.dtypesLength());
+                var seen = new HashSet<String>();
                 for (int i = 0; i < s.namesLength(); i++) {
-                    names.add(s.names(i));
+                    String name = s.names(i);
+                    if (!seen.add(name)) {
+                        // Wire contract, enforced by the reference writer ("StructLayout must
+                        // have unique field names"): a duplicate here is a crafted or corrupt
+                        // file, and the name-keyed Chunk API would silently drop a column.
+                        throw new VortexException("duplicate field name in file schema: " + name);
+                    }
+                    names.add(name);
                 }
                 for (int i = 0; i < s.dtypesLength(); i++) {
                     types.add(convertDType(s.dtypes(i), depth + 1));

reader/src/test/java/io/github/dfa1/vortex/reader/PostscriptParserDTypeGuardsTest.java

@@ -0,0 +1,92 @@
+package io.github.dfa1.vortex.reader;
+
+import io.github.dfa1.vortex.core.error.VortexException;
+import io.github.dfa1.vortex.core.fbs.FbsArraySpec;
+import io.github.dfa1.vortex.core.fbs.FbsBuilder;
+import io.github.dfa1.vortex.core.fbs.FbsDType;
+import io.github.dfa1.vortex.core.fbs.FbsFooter;
+import io.github.dfa1.vortex.core.fbs.FbsLayout;
+import io.github.dfa1.vortex.core.fbs.FbsLayoutSpec;
+import io.github.dfa1.vortex.core.fbs.FbsNull;
+import io.github.dfa1.vortex.core.fbs.FbsStruct_;
+import io.github.dfa1.vortex.core.fbs.FbsType;
+import org.junit.jupiter.api.Test;
+
+import java.lang.foreign.MemorySegment;
+
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+/// Drives the struct-dtype guards in `PostscriptParser.convertDType` with crafted dtype
+/// FlatBuffers through the package-private [PostscriptParser#parseBlobs] — no file needed.
+///
+/// Both inputs are files the reference writer refuses to produce ("StructLayout must have
+/// unique field names") or that no writer produces at all (names/dtypes arity desync), so they
+/// only arrive crafted or corrupt — and untrusted input must fail as [VortexException], never
+/// flow into the name-keyed Chunk maps where a duplicate silently drops a column.
+class PostscriptParserDTypeGuardsTest {
+
+    @Test
+    void convertDType_duplicateStructFieldNames_throwsVortexException() {
+        // Given — a dtype blob whose root struct declares two fields named "dup"
+        MemorySegment dtype = structDType(new String[]{"dup", "dup"}, 2);
+
+        // When / Then
+        assertThatThrownBy(() -> PostscriptParser.parseBlobs(minimalFooter(), flatLayout(), dtype))
+                .isInstanceOf(VortexException.class)
+                .hasMessageContaining("duplicate field name in file schema: dup");
+    }
+
+    @Test
+    void convertDType_structNamesDtypesArityMismatch_throwsVortexException() {
+        // Given — a dtype blob declaring two field names but only one field type
+        MemorySegment dtype = structDType(new String[]{"a", "b"}, 1);
+
+        // When / Then
+        assertThatThrownBy(() -> PostscriptParser.parseBlobs(minimalFooter(), flatLayout(), dtype))
+                .isInstanceOf(VortexException.class)
+                .hasMessageContaining("names/dtypes length mismatch");
+    }
+
+    // ── FlatBuffer builders ─────────────────────────────────────────────────────
+
+    /// Builds a struct dtype blob with the given field names and `dtypeCount` null-typed fields.
+    private static MemorySegment structDType(String[] names, int dtypeCount) {
+        var fbb = new FbsBuilder(256);
+        int[] fieldOffsets = new int[dtypeCount];
+        for (int i = 0; i < dtypeCount; i++) {
+            int nul = FbsNull.createFbsNull(fbb);
+            fieldOffsets[i] = FbsDType.createFbsDType(fbb, FbsType.FbsNull, nul);
+        }
+        int[] nameOffsets = new int[names.length];
+        for (int i = 0; i < names.length; i++) {
+            nameOffsets[i] = fbb.createString(names[i]);
+        }
+        int namesVec = FbsStruct_.createNamesVector(fbb, nameOffsets);
+        int dtypesVec = FbsStruct_.createDtypesVector(fbb, fieldOffsets);
+        int struct = FbsStruct_.createFbsStruct_(fbb, namesVec, dtypesVec, false);
+        int root = FbsDType.createFbsDType(fbb, FbsType.FbsStruct_, struct);
+        fbb.finish(root);
+        return fbb.dataSegment();
+    }
+
+    private static MemorySegment minimalFooter() {
+        var fbb = new FbsBuilder(256);
+        int asv = FbsFooter.createArraySpecsVector(fbb, new int[]{
+                FbsArraySpec.createFbsArraySpec(fbb, fbb.createString("vortex.primitive"))});
+        int lsv = FbsFooter.createLayoutSpecsVector(fbb, new int[]{
+                FbsLayoutSpec.createFbsLayoutSpec(fbb, fbb.createString("vortex.flat"))});
+        FbsFooter.startSegmentSpecsVector(fbb, 0);
+        int ssv = fbb.endVector();
+        int footOff = FbsFooter.createFbsFooter(fbb, asv, lsv, ssv, 0, 0);
+        fbb.finish(footOff);
+        return fbb.dataSegment();
+    }
+
+    private static MemorySegment flatLayout() {
+        var fbb = new FbsBuilder(128);
+        int segV = FbsLayout.createSegmentsVector(fbb, new long[]{0});
+        int off = FbsLayout.createFbsLayout(fbb, 0, 1L, 0, 0, segV);
+        FbsLayout.finishFbsLayoutBuffer(fbb, off);
+        return fbb.dataSegment();
+    }
+}