Skip to content

Add ALP-RD to the cascade competition (#304)#307

Merged
dfa1 merged 1 commit into
mainfrom
alprd-selection
Jul 25, 2026
Merged

Add ALP-RD to the cascade competition (#304)#307
dfa1 merged 1 commit into
mainfrom
alprd-selection

Conversation

@dfa1

@dfa1 dfa1 commented Jul 25, 2026

Copy link
Copy Markdown
Owner

What

AlpRdEncodingEncoder was registered on WriteRegistry but never added to the top-level cascade codec list, so it could only appear as a child encoding — never be selected for an F64/F32 column. High-precision floats that plain ALP can't fit without an exception on nearly every row fell back to raw vortex.primitive (or dict), while the Rust reference uses vortex.alprd.

Encoding tree for nyc-311 Latitude:

vortex-jni Latitude:   vortex.alprd          vortex-java Latitude (before):  vortex.primitive  ❌ RAW
vortex-jni Longitude:  vortex.alp            vortex-java Longitude:          vortex.alp  ✅ (plain ALP fits)

How

Add AlpRdEncodingEncoder to the cascade competition next to AlpEncodingEncoder. It competes on measured size, so it only wins where it's actually smaller: Latitude flips to alprd, Longitude keeps plain alp. No wire/reader changealprd is already decodable (the reader reads Rust files that use it; the 217-test interop suite confirms).

Impact (real nyc-311 corpus)

vortex-java vs vortex-jni (1762 MB)
Before (main) 1678.86 MB 0.95×
After 1644.25 MB 0.93×

−34.6 MB — java is now 118 MB under the Rust reference on nyc-311.

Testing

  • New integration test: a high-precision F64 column (tight range, random mantissas) selects vortex.alprd via the cascade and round-trips.
  • 217 Java-writes-Rust-reads interop tests pass; full ./mvnw verify green.

Closes #304.

AlpRdEncodingEncoder was registered on WriteRegistry but never added to the
top-level cascade codec list, so it could only appear as a child encoding, never
be selected for an F64/F32 column. High-precision floats that plain ALP cannot fit
without an exception on nearly every row (nyc-311 Latitude) therefore fell back to
raw vortex.primitive (or dict), while the Rust reference uses vortex.alprd.

Add it as a top-level candidate next to ALP. It competes on measured size, so it
only wins where it is actually smaller (Latitude flips to alprd; Longitude keeps
plain alp). No wire/reader change — alprd is already decodable (the reader reads
Rust files that use it).

nyc-311 re-encode: 1678.86 MB -> 1644.25 MB (0.95x -> 0.93x vortex-jni). Round-trip
+ ALP-RD-selection integration test added; 217 Java-writes-Rust-reads interop tests
pass; full verify green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@dfa1
dfa1 merged commit 4228c30 into main Jul 25, 2026
6 checks passed
@dfa1
dfa1 deleted the alprd-selection branch July 25, 2026 11:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nyc-311: coordinate F64 columns ~34MB larger than vortex-jni (ALP-RD not selected)

1 participant