Skip to content

[core] Add tests and implement withFilter for StatisticTable#7893

Merged
JingsongLi merged 2 commits into
apache:masterfrom
heye1005:implement-with-filter-statistic-table
May 23, 2026
Merged

[core] Add tests and implement withFilter for StatisticTable#7893
JingsongLi merged 2 commits into
apache:masterfrom
heye1005:implement-with-filter-statistic-table

Conversation

@heye1005
Copy link
Copy Markdown
Contributor

@heye1005 heye1005 commented May 18, 2026

Purpose

Remove the two // TODO markers on StatisticTable

Tests

Adds StatisticTableTest

Adds StatisticTableTest and removes the two // TODO markers in
StatisticTable.
@heye1005 heye1005 force-pushed the implement-with-filter-statistic-table branch 2 times, most recently from d69a030 to afa435f Compare May 19, 2026 03:20
@heye1005
Copy link
Copy Markdown
Contributor Author

CI Failures

These appear to be unrelated flaky tests, not caused by this PR's changes:

  • MySqlSyncTableActionITCase.testAllTypes (timeout)
  • KafkaCanalSyncDatabaseActionITCase.testExpressionNow (timeout)
  • Maven Central network timeout
    Could you help re-run the failed jobs?

I also want to separate new pr to address the CDC ITCase flakiness.

@heye1005 heye1005 force-pushed the implement-with-filter-statistic-table branch from afa435f to 86742b2 Compare May 20, 2026 09:40
@heye1005 heye1005 force-pushed the implement-with-filter-statistic-table branch from 86742b2 to cba1d34 Compare May 21, 2026 12:26
Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review comment test

Copy link
Copy Markdown
Contributor

@JingsongLi JingsongLi left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review: StatisticTable withFilter implementation

Overall this is a clean, well-structured change that resolves the TODO markers and adds solid test coverage. A few observations:

Correctness

Filter-before-projection ordering is correct. The postFilter is applied to the full-schema row produced by toRow(), and projection via ProjectedRow happens afterward. This ensures predicate column indices align with TABLE_TYPE rather than the projected readType.

Scan-level withFilter is intentionally a no-op -- since StatisticTable always produces at most one split, there is nothing to prune at the scan layer. The actual row-level filtering is delegated to StatisticRead.withFilter, which is the right approach for this table.

Minor suggestion

The Iterators.filter(rows, postFilter::test) call: since Statistics yields at most one row, the practical effect of the filter is a simple keep-or-discard decision. The current implementation is fine and consistent with how other system tables handle filtering.

Tests

Good coverage: empty statistics, basic read, predicate hit/miss on snapshot_id, and range predicate on merged_record_count. One optional enhancement for the future: a test combining withFilter and withProjection together to validate the filter-then-project ordering invariant.

Summary

The change is correct and well-tested. LGTM.

@JingsongLi
Copy link
Copy Markdown
Contributor

+1

@JingsongLi JingsongLi merged commit 2902aa2 into apache:master May 23, 2026
9 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants