Skip to content

HIVE-29028: Iceberg: Implement auto compaction #5886

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 3, 2025

Conversation

difin
Copy link
Contributor

@difin difin commented Jun 21, 2025

What changes were proposed in this pull request?

Iceberg auto compaction. In this initial version, IcebergInitiator remembers the last checked snapshot id of all Iceberg tables and it compares the current snapshot id to the cached snapshot id to determine if further evaluation is needed.

Why are the changes needed?

Currently, Iceberg compaction can only be triggered manually.
This PR introduces automatic Iceberg compaction which periodically checks all Iceberg tables and schedules compactions when necessary.

Does this PR introduce any user-facing change?

No

How was this patch tested?

New unit tests, precommit tests.

@deniskuzZ
Copy link
Member

plenty of reactions, but no reviews :)

Copy link
Contributor

@okumin okumin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly +1. I left some minor comments

Copy link
Contributor

@okumin okumin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing the remaining issues!

Copy link

sonarqubecloud bot commented Jul 2, 2025

@difin
Copy link
Contributor Author

difin commented Jul 3, 2025

Thanks, @deniskuzZ and @okumin for code reviews!

@difin difin merged commit 279996b into apache:master Jul 3, 2025
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants