Skip to content

[lake/hudi] Add Hudi Flink union read log table IT#3528

Merged
luoyuxia merged 1 commit into
apache:mainfrom
fhan688:Add-Hudi-Flink-union-read-log-table-IT
Jun 25, 2026
Merged

[lake/hudi] Add Hudi Flink union read log table IT#3528
luoyuxia merged 1 commit into
apache:mainfrom
fhan688:Add-Hudi-Flink-union-read-log-table-IT

Conversation

@fhan688

@fhan688 fhan688 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Purpose

Linked issue: #3284

This PR adds integration tests for Hudi log table union read based on the Flink union read test base introduced in #3522.

The tests verify that Fluss can read tiered Hudi snapshot data together with newly appended Fluss log data for datalake-enabled log tables.

Brief change log

  • Add FlinkUnionReadLogTableITCase for Hudi log table union read.
  • Verify batch full-type reads for both partitioned and non-partitioned log tables.
  • Verify union read after the tiering job is canceled and new Fluss log records are appended.
  • Verify projection pushdown and partition filter pushdown.
  • Verify streaming union read with partition discovery.
  • Verify savepoint-based failover and restore for a streaming insert job reading from a Hudi-backed log table.
  • Reuse the Hudi Flink union read test base from [lake/hudi] Add Flink union read test base for Hudi #3522 and enable checkpointing for the tiering job.

Tests

  • mvn -pl fluss-lake/fluss-lake-hudi -Dcheckstyle.skip=true spotless:apply
  • mvn -pl fluss-client,fluss-lake/fluss-lake-hudi -am -DskipITs -Dcheckstyle.skip=true -DfailIfNoTests=false test-compile
  • mvn -pl fluss-lake/fluss-lake-hudi -am -DskipITs=false -Dcheckstyle.skip=true -DfailIfNoTests=false -Dtest=FlinkUnionReadLogTableITCase test

API and Format

No API or storage format changes.

Documentation

No documentation changes. This PR only adds integration test coverage.

@luoyuxia luoyuxia left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@fhan688 Thanks. LGTM

@luoyuxia luoyuxia merged commit fbde73c into apache:main Jun 25, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants