Skip to content

[MINOR] precombine key needs to be strictly larger#9800

Closed
envomp wants to merge 5 commits intoapache:masterfrom
envomp:minor_precombine
Closed

[MINOR] precombine key needs to be strictly larger#9800
envomp wants to merge 5 commits intoapache:masterfrom
envomp:minor_precombine

Conversation

@envomp
Copy link
Contributor

@envomp envomp commented Sep 28, 2023

Change Logs

Before identical duplicates were overwritten when using DefaultHoodieRecordPayload. Kafka often includes identical duplicates, so ignoring the update is sometimes more performant than updating it.

Performance improvement is minor, so it might not be worth the potential risk. In that case feel free to close the PR.

Impact

Workflow for users, who publish all actual updates with same preCombine value and DefaultHoodieRecordPayload strategy, will break.

Risk level (write none, low medium or high below)

low

Documentation Update

Describe any necessary documentation update if there is any new feature, config, or user-facing change

  • The config description must be updated if new configs are added or the default value of the configs are changed
  • Any new feature or user-facing change requires updating the Hudi website. Please create a Jira ticket, attach the
    ticket number here and follow the instruction to make
    changes to the website.

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@envomp envomp closed this Sep 28, 2023
@envomp envomp deleted the minor_precombine branch September 28, 2023 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant