v0.7.1
Install: pip install oxihipo==0.7.1
A single-purpose release: undo 0.7.0's split-codec format-version bump,
which broke the C++ and Java implementations of those codecs. The composite fix
0.7.0 shipped is kept in full.
Fixed
-
Lz4PerBankandLz4PerColumnfiles written by 0.7.0 are unreadable by
hipo-cppandhipo-java. 0.7.0 raised the on-disk
ext_format_versionfrom 2 to 3 (by-bank) and 1 to 2 (by-column) when it
appended the compositeheader_sizetable. Those version numbers turn out to
be a cross-implementation contract: the
hipo-cppand
hipo-java
feature/bybank-bycolumn-compressionbranches document and implement exactly
versions 2 and 1. Measured on a JLab farm node against both:file oxihipo hipo-javahipo-cppby-bank / by-column @ 0.6.0 ✅ ✅ ✅ by-bank / by-column @ 0.7.0 ✅ ❌ failed to decode ByBank record section❌ segfault by-bank / by-column @ 0.7.1 ✅ ✅ ✅ All three now produce byte-identical checksums on the same files, array
(T#N) columns included.The bump was never necessary. The
header_sizetable is appended after
every other directory table, so a reader that predates it never looks that
far — proven by patching only the version byte back on a 0.7.0 file, after
which both other implementations read it perfectly. The library now detects
the table by directory length instead of by the version byte, which is
what lets the version stay fixed while the format grows.0.7.1 still reads the version-3/2 files 0.7.0 wrote, so nothing already
written is lost. Rewrite them with 0.7.1 if you need to share them.
Added
tests/composite_codecs.rsasserts the on-diskext_format_versionis 2
(by-bank) and 1 (by-column), reading the byte back off the file. The contract
is with other codebases, so it needed a test that fails if it drifts again.
Documentation
- The claim that the split codecs are readable only by this library was wrong
and is corrected everywhere it appeared. The released C++hipo4and Java
readers do not know wire tags 6 and 7, but the branches above do.