Skip to content

Commit

Permalink
[SPARK-36814][SQL] Make class ColumnarBatch extendable
Browse files Browse the repository at this point in the history
### What changes were proposed in this pull request?
Change class ColumnarBatch to a non-final class

### Why are the changes needed?
To support better vectorized reading in multiple data source, ColumnarBatch need to be extendable. For example, To support row-level delete(  apache/iceberg#3141) in Iceberg's vectorized read, we need to filter out deleted rows in a batch, which requires ColumnarBatch to be extendable.

### Does this PR introduce _any_ user-facing change?
No

### How was this patch tested?
No test needed.

Closes #34054 from flyrain/columnarbatch-extendable.

Authored-by: Yufei Gu <yufei_gu@apple.com>
Signed-off-by: DB Tsai <d_tsai@apple.com>
  • Loading branch information
Yufei Gu authored and dbtsai committed Sep 21, 2021
1 parent d2340f8 commit 688b95b
Showing 1 changed file with 1 addition and 1 deletion.
Expand Up @@ -31,7 +31,7 @@
* the entire data loading process.
*/
@Evolving
public final class ColumnarBatch implements AutoCloseable {
public class ColumnarBatch implements AutoCloseable {
private int numRows;
private final ColumnVector[] columns;

Expand Down

0 comments on commit 688b95b

Please sign in to comment.