[C++][Parquet] Would ReadNewPage conflict with data_page_filter?

[mapleFU]

Describe the enhancement requested

parquet::ColumnReader::HasNextInternal might call ReadNewPage to check the record boundary.

  bool HasNextInternal() {
    // Either there is no data page available yet, or the data page has been
    // exhausted
    if (num_buffered_values_ == 0 || num_decoded_values_ == num_buffered_values_) {
      if (!ReadNewPage() || num_buffered_values_ == 0) {
        return false;
      }
    }
    return true;
  }

And ReadNewPage will call:

  // Advance to the next data page
  bool ReadNewPage() {
    // Loop until we find the next data page.
    while (true) {
      current_page_ = pager_->NextPage();
      if (!current_page_) {
        // EOS
        return false;
      }

When having data_page_filter, in v1 format, seems that NextPage might filter the data-page?

Component(s)

C++, Parquet

[mapleFU]

cc @emkornfield @fatemehp

I think this might not be triggered when page-index enabled, but would this be a problem in v1 page with cross-page record?

[fatemehp]

data page filter should not be set with V1 repeated fields. This is because we do not have num_rows in DataPageStats for V1 repeated fields. So the stats that the callback is called with is not helpful in that case anyways. We should add a comment in the code. Note that the set_data_page_filter API is marked experimental.

…

On Fri, Nov 24, 2023 at 10:35 PM mwish ***@***.***> wrote: cc @emkornfield <https://github.com/emkornfield> @fatemehp <https://github.com/fatemehp> I think this might not be triggered when page-index enabled, but would this be a problem in v1 page with cross-page record? — Reply to this email directly, view it on GitHub <#38880 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABOHYRYD2QCZJN3NX5C2YRDYGGGTRAVCNFSM6AAAAAA7Z4K2ZOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTQMRWGIZTCMJZGY> . You are receiving this because you were mentioned.Message ID: ***@***.***>