Skip to content

Commit

Permalink
GH-38725: [Java] decompression in Lz4CompressionCodec.java does not s…
Browse files Browse the repository at this point in the history
…et writer index (#38840)

### Rationale for this change

The `doDecompress` function in `Lz4CompressionCodec` misses writing the index when it is compared with the functionality in `ZstdCompressionCodec`. This PR fixes that issue. 

### What changes are included in this PR?

Writes the index for the decompressed ArrowBuf. 

### Are these changes tested?

No

### Are there any user-facing changes?

No
* Closes: #38725

Lead-authored-by: Vibhatha Lakmal Abeykoon <vibhatha@gmail.com>
Co-authored-by: vibhatha <vibhatha@gmail.com>
Signed-off-by: David Li <li.davidm96@gmail.com>
  • Loading branch information
vibhatha and vibhatha committed Dec 21, 2023
1 parent 2308cdf commit 596259e
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,7 @@ protected ArrowBuf doDecompress(BufferAllocator allocator, ArrowBuf compressedBu
byte[] outBytes = out.toByteArray();
ArrowBuf decompressedBuffer = allocator.buffer(outBytes.length);
decompressedBuffer.setBytes(/*index=*/0, outBytes);
decompressedBuffer.writerIndex(decompressedLength);
return decompressedBuffer;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -117,6 +117,12 @@ private List<ArrowBuf> deCompressBuffers(CompressionCodec codec, List<ArrowBuf>
return outputBuffers;
}

private void assertWriterIndex(List<ArrowBuf> decompressedBuffers) {
for (ArrowBuf decompressedBuf : decompressedBuffers) {
assertTrue(decompressedBuf.writerIndex() > 0);
}
}

@ParameterizedTest
@MethodSource("codecs")
void testCompressFixedWidthBuffers(int vectorLength, CompressionCodec codec) throws Exception {
Expand All @@ -139,6 +145,7 @@ void testCompressFixedWidthBuffers(int vectorLength, CompressionCodec codec) thr
List<ArrowBuf> decompressedBuffers = deCompressBuffers(codec, compressedBuffers);

assertEquals(2, decompressedBuffers.size());
assertWriterIndex(decompressedBuffers);

// orchestrate new vector
IntVector newVec = new IntVector("new vec", allocator);
Expand Down Expand Up @@ -180,6 +187,7 @@ void testCompressVariableWidthBuffers(int vectorLength, CompressionCodec codec)
List<ArrowBuf> decompressedBuffers = deCompressBuffers(codec, compressedBuffers);

assertEquals(3, decompressedBuffers.size());
assertWriterIndex(decompressedBuffers);

// orchestrate new vector
VarCharVector newVec = new VarCharVector("new vec", allocator);
Expand Down

0 comments on commit 596259e

Please sign in to comment.