Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HUDI-3720] Fix the logic of reattempting pending rollback #5148

Merged

Conversation

yihua
Copy link
Contributor

@yihua yihua commented Mar 27, 2022

What is the purpose of the pull request

When reattempting a pending rollback and the instant to rollback does not exist in the active timeline anymore due to a failure just before the rollback was transitioned from inflight to complete, such rollback cannot successfully be completed. This PR fixes the logic so that in such case the rollback can be reattempted successfully.

Brief change log

  • Changes how the rollback is executed in different scenarios in BaseHoodieWriteClient.
  • Adds unit tests in TestClientRollback.

Verify this pull request

This change added tests in TestClientRollback and is verified as follows:

  • Run deltastreamer continuous mode writing MOR table. Fail the deltastreamer and a rollback before markers are deleted or just before the rollback is transitioned from inflight to complete. Rerun the deltastreamer. Verified that the same rollback can finish successfully (which was failing to retry before this change), markers are deleted, and the rollback metadata is correct.

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@apache apache deleted a comment from hudi-bot Mar 28, 2022
@yihua yihua changed the title [WIP][HUDI-3720] Fix the logic of reattempting pending rollback [HUDI-3720] Fix the logic of reattempting pending rollback Mar 28, 2022
@hudi-bot
Copy link

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants