Fix patch_chunk index OOB when slicing ALP arrays mid-chunk#7354
Fix patch_chunk index OOB when slicing ALP arrays mid-chunk#7354robert3005 merged 2 commits intovortex-data:developfrom
Conversation
Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com>
Merging this PR will degrade performance by 23.81%
Performance Changes
Comparing Footnotes
|
There was a problem hiding this comment.
offset_within_chunk has to always be used with saturating sub, we don't know where exactly the indicies fall in the chunk. I thought that this wasn't necessary for subsequent chunks after the current chunk but we don't know if chunks have any patches at all
Signed-off-by: Abanoub Doss <abanoub.doss@gmail.com>
Summary
patch_chunkpanics with index-out-of-bounds when decompressing sliced ALP arrays with patches. The slice boundary must fall mid-chunk (non-1024-aligned) to trigger it.Root cause:
Patches::slice()sliceschunk_offsetsat chunk granularity butindices/valuesat element granularity. When a slice ends mid-chunk,patches_end_idxcan exceedpatches_indices.len(). The start index already usedsaturating_sub— the end index didn't.Fix:
.saturating_sub(offset_within_chunk).min(patches_indices.len())on the end index computation inpatch_chunk,search_index_chunked, andsearch_index_chunked_batch.AI disclosure: Root cause analysis, fix implementation, and test writing were done with Claude Code under my direction and review. I identified the bug during development benchmarking at 100M rows and verified the fix against my workload.
Testing
patch_chunkwith inputs that OOB without the fix.