Skip to content

Support HoodieSourceSplitComparator for better watermark alignment #18019

@HuangZhenQiu

Description

@HuangZhenQiu

Task Description

What needs to be done:
Add HoodieSourceSplitComparator that sort hoodie split with base file commit time.

Why this task is needed:
By supporting this feature, we may guarantee files committed with the same commit time will be processed in sequentially for better watermark alignment

Task Type

Code improvement/refactoring

Related Issues

Parent feature issue: (if applicable )
Related issues:
NOTE: Use Relationships button to add parent/blocking issues after issue is created.

Metadata

Metadata

Assignees

Labels

type:devtaskDevelopment tasks and maintenance work

Type

No type

Projects

Status

In progress

Relationships

None yet

Development

No branches or pull requests

Issue actions