Skip to content

refactor: consolidate common utility classes for Flink CDC read#18436

Merged
danny0405 merged 2 commits into
apache:masterfrom
HuangZhenQiu:refactor-source-v2
Apr 3, 2026
Merged

refactor: consolidate common utility classes for Flink CDC read#18436
danny0405 merged 2 commits into
apache:masterfrom
HuangZhenQiu:refactor-source-v2

Conversation

@HuangZhenQiu
Copy link
Copy Markdown
Member

Describe the issue this Pull Request addresses

Consolidate common utility classes for Flink CDC read to reduce duplicated code

Summary and Changelog

  1. refactor out CdcImageManager class
  2. refactor out all CDC related iterators into CdcIterators

Impact

none

Risk Level

none

Documentation Update

none

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable

@github-actions github-actions Bot added the size:XL PR with lines of changes > 1000 label Apr 1, 2026
@HuangZhenQiu HuangZhenQiu changed the title refact: consolidate common utility classes for Flink CDC read refactor: consolidate common utility classes for Flink CDC read Apr 1, 2026
Copy link
Copy Markdown
Collaborator

@cshuo cshuo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thks for the contribution, left some comments.

}
}

private ClosableIterator<RowData> createRecordIterator(
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible refactor this method as an util method in CdcIterators, since it's almost same with CdcInputFormat#getRecordIterator

@codecov-commenter
Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 71.22093% with 99 lines in your changes missing coverage. Please review.
✅ Project coverage is 68.70%. Comparing base (3e1d300) to head (f035c44).
⚠️ Report is 7 commits behind head on master.

Files with missing lines Patch % Lines
...org/apache/hudi/table/format/cdc/CdcIterators.java 78.66% 37 Missing and 11 partials ⚠️
.../apache/hudi/table/format/cdc/CdcImageManager.java 64.17% 21 Missing and 3 partials ⚠️
.../reader/function/HoodieCdcSplitReaderFunction.java 12.50% 21 Missing ⚠️
...g/apache/hudi/table/format/cdc/CdcInputFormat.java 78.57% 6 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##             master   #18436      +/-   ##
============================================
+ Coverage     68.54%   68.70%   +0.15%     
- Complexity    27975    28044      +69     
============================================
  Files          2442     2446       +4     
  Lines        134530   134481      -49     
  Branches      16251    16264      +13     
============================================
+ Hits          92218    92390     +172     
+ Misses        35038    34810     -228     
- Partials       7274     7281       +7     
Flag Coverage Δ
common-and-other-modules 44.46% <71.22%> (+0.10%) ⬆️
hadoop-mr-java-client 44.93% <ø> (-0.05%) ⬇️
spark-client-hadoop-common 48.46% <ø> (+0.07%) ⬆️
spark-java-tests 48.81% <ø> (+0.02%) ⬆️
spark-scala-tests 45.63% <ø> (-0.03%) ⬇️
utilities 38.37% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...g/apache/hudi/table/format/cdc/CdcInputFormat.java 76.71% <78.57%> (+2.21%) ⬆️
.../reader/function/HoodieCdcSplitReaderFunction.java 16.66% <12.50%> (+14.15%) ⬆️
.../apache/hudi/table/format/cdc/CdcImageManager.java 64.17% <64.17%> (ø)
...org/apache/hudi/table/format/cdc/CdcIterators.java 78.66% <78.66%> (ø)

... and 34 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@hudi-bot
Copy link
Copy Markdown
Collaborator

hudi-bot commented Apr 3, 2026

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@danny0405 danny0405 merged commit 57c3f63 into apache:master Apr 3, 2026
56 checks passed
rahil-c pushed a commit to rahil-c/hudi that referenced this pull request Apr 20, 2026
…he#18436)

* refactor: consolidate common utility classes for Flink CDC read
* resolve shuo's comments
dwshmilyss pushed a commit to dwshmilyss/hudi that referenced this pull request May 21, 2026
…he#18436)

* refactor: consolidate common utility classes for Flink CDC read
* resolve shuo's comments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL PR with lines of changes > 1000

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants