Skip to content

[HUDI-6369] Fix spacial curve with sample strategy fails when 0 or 1 rows only is incoming#9053

Merged
bvaradar merged 2 commits intoapache:masterfrom
parisni:HUDI-6369/fix-zorder
Oct 18, 2023
Merged

[HUDI-6369] Fix spacial curve with sample strategy fails when 0 or 1 rows only is incoming#9053
bvaradar merged 2 commits intoapache:masterfrom
parisni:HUDI-6369/fix-zorder

Conversation

@parisni
Copy link
Contributor

@parisni parisni commented Jun 26, 2023 •
edited by bvaradar
Loading

Change Logs

Fixes #8934

see https://issues.apache.org/jira/browse/HUDI-6369

Impact

Fix corner case for z-order clustering

Risk level (write none, low medium or high below)

low

Documentation Update

N/A

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@parisni parisni force-pushed the HUDI-6369/fix-zorder branch from a7c8776 to 7935c3a Compare June 28, 2023 22:25
@parisni parisni requested a review from boneanxs June 28, 2023 22:29
@parisni parisni requested a review from boneanxs June 29, 2023 19:42
boneanxs
boneanxs reviewed Jun 30, 2023
View reviewed changes
Copy link
Contributor

@boneanxs boneanxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you pls look into the test failure? It appeals new added test has errors. Also, this test can moved to TestLayoutOptimization.

Currently, we lack tests that cover the sortDataFrameBySampleSupportAllTypes function. It would be highly beneficial if you could include it as well.

@parisni
Copy link
Contributor Author

parisni commented Jun 30, 2023

Currently, we lack tests that cover the sortDataFrameBySampleSupportAllTypes function. It would be highly beneficial if you could include it as well.

Agreed, fill free to submit a patch, I am in vacation for a week

@parisni parisni requested a review from boneanxs June 30, 2023 10:17
@parisni
Copy link
Contributor Author

parisni commented Jul 10, 2023

Currently, we lack tests that cover the sortDataFrameBySampleSupportAllTypes function. It would be highly beneficial if you could include it as well

  • removed datasource test
  • introduced new test for both sort methods to cover case
  • verified test fails w/o the fix

@boneanxs

boneanxs
boneanxs approved these changes Jul 11, 2023
View reviewed changes
Copy link
Contributor

@boneanxs boneanxs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall LGTM, @danny0405 could you please also review this? Thanks!

@parisni parisni requested a review from boneanxs July 11, 2023 08:48
@bvaradar bvaradar self-assigned this Oct 4, 2023
@bvaradar bvaradar added the type:bug Bug reports and fixes label Oct 4, 2023
@bvaradar bvaradar force-pushed the HUDI-6369/fix-zorder branch from ff5cd07 to 8caf378 Compare October 18, 2023 04:15
@bvaradar
Copy link
Contributor

bvaradar commented Oct 18, 2023

Looks good to me. Resurrecting this PR with rebase and minor test class rename.

@hudi-bot
Copy link
Collaborator

hudi-bot commented Oct 18, 2023

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@bvaradar bvaradar merged commit 2dc1c60 into apache:master Oct 18, 2023
nsivabalan pushed a commit that referenced this pull request Nov 21, 2023
@parisni @bvaradar @nsivabalan
[HUDI-6369] Fix spacial curve with sample strategy fails when 0 or 1 …
871f8b7 
…rows only is incoming (#9053)

* [HUDI-6369] Fix spacial when empty or 1 row df

* Rename unit test to follow conventions

---------

Co-authored-by: Balaji Varadarajan <vbalaji@apache.org>
@hudi-bot hudi-bot mentioned this pull request Dec 9, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bvaradar bvaradar bvaradar approved these changes

@boneanxs boneanxs Awaiting requested review from boneanxs

Assignees

@bvaradar bvaradar

Labels

type:bug Bug reports and fixes

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[SUPPORT] Spacial curve with sample strategy fails when 0 or 1 rows only is incoming

4 participants

@parisni @bvaradar @hudi-bot @boneanxs