Skip to content

Commit

Permalink
ORC-1696: Fix ClassCastException when reading avro decimal type in be…
Browse files Browse the repository at this point in the history
…chmark

### What changes were proposed in this pull request?
This PR aims to fix `ClassCastException` when reading avro decimal type in bechmark.

### Why are the changes needed?
ORC-1191 Forcing `object` to `double`, but object type is `ByteBuffer`, which causes scan to fail.

```bash
 java -jar core/target/orc-benchmarks-core-*-uber.jar scan data
```

```java
Exception in thread "main" java.lang.ClassCastException: class java.nio.HeapByteBuffer cannot be cast to class java.lang.Double (java.nio.HeapByteBuffer and java.lang.Double are in module java.base of loader 'bootstrap')
	at org.apache.orc.bench.core.convert.avro.AvroReader$DecimalConverter.convert(AvroReader.java:204)
	at org.apache.orc.bench.core.convert.avro.AvroReader.nextBatch(AvroReader.java:69)
	at org.apache.orc.bench.core.convert.ScanVariants.run(ScanVariants.java:92)
	at org.apache.orc.bench.core.Driver.main(Driver.java:64)
```

### How was this patch tested?
local test
```bash
 java -jar core/target/orc-benchmarks-core-*-uber.jar scan data
```
output
```
data/generated/taxi/avro.snappy rows: 22758236 batches: 22225
```

### Was this patch authored or co-authored using generative AI tooling?
No

Closes #1898 from cxzl25/ORC-1696.

Authored-by: sychen <sychen@ctrip.com>
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
(cherry picked from commit d4f13dc)
Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
  • Loading branch information
cxzl25 authored and dongjoon-hyun committed Apr 22, 2024
1 parent 2d6a3d1 commit e21eafa
Showing 1 changed file with 13 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
import org.apache.orc.bench.core.convert.BatchReader;

import java.io.IOException;
import java.math.BigInteger;
import java.nio.ByteBuffer;
import java.util.List;

Expand Down Expand Up @@ -201,7 +202,11 @@ public void convert(ColumnVector cv, int row, Object value) {
cv.isNull[row] = true;
} else {
DecimalColumnVector tc = (DecimalColumnVector) cv;
tc.vector[row].set(HiveDecimal.create(Math.round((double) value * multiplier)));
if (value instanceof ByteBuffer) {
tc.vector[row].set(getHiveDecimalFromByteBuffer((ByteBuffer) value, scale));
} else {
tc.vector[row].set(HiveDecimal.create(Math.round((double) value * multiplier)));
}
}
}
}
Expand Down Expand Up @@ -289,6 +294,13 @@ static AvroConverter createConverter(TypeDescription types) {
}
}

static HiveDecimal getHiveDecimalFromByteBuffer(ByteBuffer byteBuffer,
int scale) {
byte[] result = getBytesFromByteBuffer(byteBuffer);
HiveDecimal dec = HiveDecimal.create(new BigInteger(result), scale);
return dec;
}

static byte[] getBytesFromByteBuffer(ByteBuffer byteBuffer) {
byteBuffer.rewind();
byte[] result = new byte[byteBuffer.limit()];
Expand Down

0 comments on commit e21eafa

Please sign in to comment.