From 688b95b136571fa559f26e6582fc3fc296f9d1bf Mon Sep 17 00:00:00 2001 From: Yufei Gu Date: Tue, 21 Sep 2021 15:24:55 +0000 Subject: [PATCH] [SPARK-36814][SQL] Make class ColumnarBatch extendable ### What changes were proposed in this pull request? Change class ColumnarBatch to a non-final class ### Why are the changes needed? To support better vectorized reading in multiple data source, ColumnarBatch need to be extendable. For example, To support row-level delete( https://github.com/apache/iceberg/issues/3141) in Iceberg's vectorized read, we need to filter out deleted rows in a batch, which requires ColumnarBatch to be extendable. ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No test needed. Closes #34054 from flyrain/columnarbatch-extendable. Authored-by: Yufei Gu Signed-off-by: DB Tsai --- .../java/org/apache/spark/sql/vectorized/ColumnarBatch.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java b/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java index a2feac869ece6..b5c3ed79f14a8 100644 --- a/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java +++ b/sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ColumnarBatch.java @@ -31,7 +31,7 @@ * the entire data loading process. */ @Evolving -public final class ColumnarBatch implements AutoCloseable { +public class ColumnarBatch implements AutoCloseable { private int numRows; private final ColumnVector[] columns;