Skip to content

test(client): improve native log and CDC write coverage - #19353

Merged
danny0405 merged 3 commits into
apache:masterfrom
cshuo:codex/hudi-19352-native-log-cdc-coverage
Jul 28, 2026
Merged

test(client): improve native log and CDC write coverage#19353
danny0405 merged 3 commits into
apache:masterfrom
cshuo:codex/hudi-19352-native-log-cdc-coverage

Conversation

@cshuo

@cshuo cshuo commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Describe the issue this Pull Request addresses

Closes #19352.

The native log format and CDC write handles in hudi-client-common lacked direct test coverage. This left record routing, file rollover, CDC supplemental logging modes, merge ordering, statistics publication, and representative failure handling insufficiently protected.

This PR adds focused tests and raises every class in scope above the 70% line-coverage target.

Class Before After
HoodieNativeLogFormatWriter 0.0% 84.7%
HoodieNativeLogAppendHandle 0.0% 71.7%
FileGroupReaderBasedNativeLogAppendHandle 0.0% 88.7%
HoodieNativeCDCLogger 0.0% 91.4%
HoodieNativeCDCFileWriter 0.0% 98.2%
HoodieAvroNativeCDCLogger 0.0% 90.9%
HoodieSortedMergeHandle 0.0% 77.5%
HoodieMergeHandleWithChangeLog 0.0% 83.9%

Summary and Changelog

Commit 1: test(client): improve native log and CDC write coverage (4956c8de4d3)

  • Add native log append-handle tests for data/delete routing, rollover, write-count accounting, configured record keys, ignored records, lifecycle behavior, and flush failures.
  • Cover file-group-reader append behavior, instant and read-stat propagation, write-status annotation, and failure wrapping.
  • Exercise Avro and generic native CDC loggers for all supplemental logging modes across insert, update, delete, pending-record, retraction, and close paths.
  • Extend native CDC file-writer coverage for file rolling, version collision handling, creation callbacks, partitioned/non-partitioned statistics, and I/O failures.
  • Cover sorted merge ordering and pending inserts, plus insert/update CDC emission and status publication for change-log merge handles.

Impact

  • Functional impact: None. This is a test-only change and does not modify production behavior.
  • Maintainability: Protects native log and CDC write paths against regressions with focused behavioral assertions.
  • Extensibility: Provides test fixtures for adding native log formats, CDC modes, and merge-handle behavior.

Risk Level

Low. The change only adds and extends unit tests. The targeted suite, Checkstyle, and Apache RAT all pass.

Documentation Update

None.

Contributor's checklist

  • Read through contributor's guide
  • Enough context is provided in the sections above
  • Adequate tests were added if applicable
  • mvn -pl hudi-client/hudi-client-common -am -DskipTests -DskipITs -DskipSparkTests -DskipScalaTests checkstyle:check
  • Native log/CDC targeted suite: 28 tests, 0 failures, 0 errors, 0 skipped

@cshuo cshuo changed the title [hudi-client] Improve native log and CDC write handle test coverage test(client): improve native log and CDC write coverage Jul 23, 2026
@cshuo
cshuo marked this pull request as ready for review July 23, 2026 03:35
@github-actions github-actions Bot added the size:XL PR with lines of changes > 1000 label Jul 23, 2026

@danny0405 danny0405 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I found two gaps where the new tests can pass without validating the behavior named in the test and PR.

@cshuo cshuo left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments addressed.

@danny0405
danny0405 enabled auto-merge (squash) July 27, 2026 03:29
@voonhous

Copy link
Copy Markdown
Member

@cshuo I think there are some failing CI tests, can you please help to fix them? Thank you!

auto-merge was automatically disabled July 28, 2026 06:39

Head branch was pushed to by a user without write access

@cshuo
cshuo force-pushed the codex/hudi-19352-native-log-cdc-coverage branch from 5d8636b to d4fca99 Compare July 28, 2026 06:39
@codecov-commenter

codecov-commenter commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 72.25%. Comparing base (d56e674) to head (34379f1).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files
@@             Coverage Diff              @@
##             master   #19353      +/-   ##
============================================
+ Coverage     72.19%   72.25%   +0.06%     
- Complexity    32595    32628      +33     
============================================
  Files          2561     2561              
  Lines        148328   148328              
  Branches      18654    18654              
============================================
+ Hits         107087   107177      +90     
+ Misses        32774    32685      -89     
+ Partials       8467     8466       -1     
Components Coverage Δ
hudi-common 81.44% <ø> (+0.02%) ⬆️
hudi-client 81.80% <ø> (+0.23%) ⬆️
hudi-flink 78.63% <ø> (+<0.01%) ⬆️
hudi-spark-datasource 58.66% <ø> (ø)
hudi-utilities 70.42% <ø> (ø)
hudi-cli 15.32% <ø> (ø)
hudi-hadoop 63.49% <ø> (ø)
hudi-sync 70.62% <ø> (-0.03%) ⬇️
hudi-io 79.57% <ø> (ø)
hudi-timeline-service 83.54% <ø> (+0.09%) ⬆️
hudi-cloud 64.00% <ø> (ø)
hudi-kafka-connect 53.96% <ø> (+0.76%) ⬆️
Flag Coverage Δ
common-and-other-modules 46.79% <ø> (+0.08%) ⬆️
flink-integration-tests 47.70% <ø> (+<0.01%) ⬆️
hadoop-mr-java-client 43.31% <ø> (-0.01%) ⬇️
integration-tests 13.59% <ø> (+<0.01%) ⬆️
spark-client-hadoop-common 48.69% <ø> (-0.01%) ⬇️
spark-java-tests 49.34% <ø> (+0.01%) ⬆️
spark-scala-tests 44.86% <ø> (-0.01%) ⬇️
utilities 36.48% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 23 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@hudi-bot

Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@danny0405
danny0405 merged commit 3f2de46 into apache:master Jul 28, 2026
75 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XL PR with lines of changes > 1000

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[hudi-client] Improve native log and CDC write handle test coverage

5 participants