Repository navigation
Defects found by model-based property tests of 2.2.0
(oaustegard/experiments/remex-proptest),
each covered by tests/test_property_findings.py.
Fixed
.npzfiles read back the same codes in both container classes.CompressedVectors.savepacks one bit stream andPackedVectors.savepads each row to a byte. Whend * bitsis not a multiple of 8 the two differ, and reading one class's file with the other returned wrong codes without an error. Both writers now record alayoutkey and both readers honour it. Files without the key are told apart by their byte count. Where the two sizes coincide (for example d=7, n=5 at 1 or 3 bits) an old file is read the way the reading class always read it, which is correct for a file that class wrote.Quantizer.searchruns the bits and mean guards on a warm container. A cache filled by another quantizer skipped them, so withrenorm=Falseor in scalar mode a mismatched reader returned plausible scores instead of raising.search_batchalready ran them.- A pickled or deep-copied rht
Quantizerno longer segfaults.RHTOperatorcached raw buffer addresses; it now pickles as(d, seed). - A rejected first batch leaves a model-less
RemexIndexusable.add([])raisedZeroDivisionErrorand setdim=0, after which everyaddfailed. Empty batches raiseValueError, anddimis set only once a batch is accepted. RemexIndex.loadgoes throughopen_npz: it finds the filesavewrote and names a damaged one.IVFCoarseIndexruns the quantizer's guards when it is built, instead of hashing codes of another width into a wrong index.save_arrowrefuses extra metadata namedd,bits,n,rotation,seed,scaleormean, which could relabel the encoding.- A damaged
.npzmember raises the namedValueError, not a zip, zlib or I/O error from the lazy member read. - A seed of 2**63 or more can be saved to
.npz(as a decimal string). PackedVectors.loadreads an indices-only.npz, asCompressedVectors.loaddoes.- Vectors shorter than 1e-8 are coded by their direction. A 1e-8 floor on the norm shrank them into the central cells; only an exact zero is guarded now. Codes of such vectors change; no other codes do.
_score_vectorsanddecodeaccept uncentredPackedVectors; they read.indicesand raisedAttributeErrorexcept on centred codes.- IVF's flat limit is byte-identical to the flat paths for centred codes.
nprobe = 2**n_bitsadded the query offset before the row bias, and the flat paths add it after, so scores differed by an ulp. IVFCoarseIndex.candidate_indices(nprobe=1)returns a copy, not a view through which a caller could overwrite the index.- Smaller guards: a
scalethat makes the float32 codebook degenerate is refused;coordinate_sigmarefusesd <= 0;assign_codesmatchessearchsortedfor integer, byte-swapped and mixed-precision inputs and checks the boundary count; a non-finite row no longer raises a spuriousAnisotropyWarning;corpus_meanof an empty corpus raises.
Known, not changed
- The 8-bit Lloyd-Max table stops at 300 iterations, with MSE 20% above the fixed point. Converging it would change every stored 8-bit code.
rhtat d=2 gives the same decode for every seed, and at small even d a few seeds leave a coordinate unmixed.- The constant
q . muis added in float32 before top-k, so with||mu|| / ||x - mu||around 1e5 or more it can round two rows into a tie.