Skip to content

minor: Track Parquet rows and pages matched when the page index is skipped#22085

Merged
kumarUjjawal merged 5 commits into
apache:mainfrom
nuno-faria:fix_parquet_page_index
May 12, 2026
Merged

minor: Track Parquet rows and pages matched when the page index is skipped#22085
kumarUjjawal merged 5 commits into
apache:mainfrom
nuno-faria:fix_parquet_page_index

Conversation

@nuno-faria
Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

If page index pruning is attempted but fails, report that all rows/pages have matched, instead of returning 0 (see #22031 (comment)).

What changes are included in this PR?

  • Track the number of matched rows and pages independently of the result of the predicate pruning.

Are these changes tested?

Existing tests. I'm not sure how to generate Parquet files that trigger this directly.

Are there any user-facing changes?

No.

cc: @adriangb @2010YOUY01

@github-actions github-actions Bot added the datasource Changes to the datasource crate label May 9, 2026
Comment thread datafusion/datasource-parquet/src/page_filter.rs Outdated
@kumarUjjawal kumarUjjawal enabled auto-merge May 12, 2026 11:50
@kumarUjjawal
Copy link
Copy Markdown
Contributor

Thank you @nuno-faria and @xudong963

@kumarUjjawal kumarUjjawal added this pull request to the merge queue May 12, 2026
Merged via the queue into apache:main with commit b76970b May 12, 2026
35 checks passed
@nuno-faria nuno-faria deleted the fix_parquet_page_index branch May 12, 2026 12:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

datasource Changes to the datasource crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants