feat(scan): support data evolution row ID range filter#207

Closed

XiaoHongbo-Hope wants to merge 23 commits into

apache:mainfrom

XiaoHongbo-Hope:support_row_id_filter

Commits on Apr 6, 2026

support data evolution row id
XiaoHongbo-Hope
committed
clean code
XiaoHongbo-Hope
committed
fix: skip parquet-level row_ranges filtering when _ROW_ID is projected
XiaoHongbo-Hope
committed
fix: disable limit pushdown when row_ranges is set
XiaoHongbo-Hope
committed
fix: apply row_ranges post-read filter when _ROW_ID is projected
XiaoHongbo-Hope
committed
fix: skip row_ranges filtering for files without first_row_id (fail-open)
XiaoHongbo-Hope
committed
fix: return null _ROW_ID for files without first_row_id instead of faking from 0
XiaoHongbo-Hope
committed
clean code
XiaoHongbo-Hope
committed
fix: use saturating_add to avoid i64 overflow in merge_row_ranges
XiaoHongbo-Hope
committed
refactor: extract insert_column_at to deduplicate append_row_id_column and append_null_row_id_column
XiaoHongbo-Hope
committed
refactor: align _ROW_ID with Java - use pre-computed row IDs with RowSelection instead of skipping IO filtering
XiaoHongbo-Hope
committed
style: remove unnecessary comments
XiaoHongbo-Hope
committed
fix: guard against clamped_to underflow and extract attach_row_id helper
XiaoHongbo-Hope
committed
fix: NULL-fill branch in merge_files_by_columns respects row_ranges
XiaoHongbo-Hope
committed
perf: lazy row ID computation when no row_ranges, avoid full-file allocation
XiaoHongbo-Hope
committed
fix: merge overlapping row_ranges in expand_selected_row_ids to match RowSelection
XiaoHongbo-Hope
committed
add debug_assert for merge group invariants aligned with Java checkArgument
XiaoHongbo-Hope
committed
use assert! instead of debug_assert! for merge group invariants
XiaoHongbo-Hope
committed
remove unused is_row_id_field and is_empty methods
XiaoHongbo-Hope
committed
address review: assert in RowRange::new, handle row_count==0, remove redundant merge, avoid clone in merge_row_ranges, update limit comment
XiaoHongbo-Hope
committed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(scan): support data evolution row ID range filter#207

feat(scan): support data evolution row ID range filter#207
XiaoHongbo-Hope wants to merge 23 commits into
apache:mainfrom
XiaoHongbo-Hope:support_row_id_filter

Commits on Apr 6, 2026

support data evolution row id

clean code

fix: skip parquet-level row_ranges filtering when _ROW_ID is projected

fix: disable limit pushdown when row_ranges is set

fix: apply row_ranges post-read filter when _ROW_ID is projected

fix: skip row_ranges filtering for files without first_row_id (fail-open)

fix: return null _ROW_ID for files without first_row_id instead of faking from 0

clean code

fix: use saturating_add to avoid i64 overflow in merge_row_ranges

refactor: extract insert_column_at to deduplicate append_row_id_column and append_null_row_id_column

refactor: align _ROW_ID with Java - use pre-computed row IDs with RowSelection instead of skipping IO filtering

style: remove unnecessary comments

fix: guard against clamped_to underflow and extract attach_row_id helper

fix: NULL-fill branch in merge_files_by_columns respects row_ranges

perf: lazy row ID computation when no row_ranges, avoid full-file allocation

fix: merge overlapping row_ranges in expand_selected_row_ids to match RowSelection

add debug_assert for merge group invariants aligned with Java checkArgument

use assert! instead of debug_assert! for merge group invariants

remove unused is_row_id_field and is_empty methods

address review: assert in RowRange::new, handle row_count==0, remove redundant merge, avoid clone in merge_row_ranges, update limit comment

Commits on Apr 7, 2026

support _ROW_ID in DataFusion SQL for data evolution tables

fix: quote _ROW_ID in SQL to preserve case sensitivity

fix: always pass column names to ReadBuilder to include _ROW_ID in SELECT *