Skip to content

HIVE-25257: Incorrect row order validation for query-based MAJOR comp…#2400

Merged
kuczoram merged 1 commit intoapache:masterfrom
kuczoram:HIVE-25257
Jun 18, 2021
Merged

HIVE-25257: Incorrect row order validation for query-based MAJOR comp…#2400
kuczoram merged 1 commit intoapache:masterfrom
kuczoram:HIVE-25257

Conversation

@kuczoram
Copy link
Contributor

  • Fix the row order validation for the query-based MAJOR compaction. The rows should be in increasing order by originalTransactionId, bucketProperty, rowId. But in the validation the bucketProperty is checked first, which can cause failure if there are delta directories with multiple statementIds, like after multiple merge statements.
  • Added unit test for this use-case
  • Fixed the error message printed out in the GenericUDFValidateAcidSortOrder by adding a toString() method to the WriteIdRowId inner class.

@github-actions github-actions bot requested a review from abstractdog June 16, 2021 14:34
@kuczoram kuczoram requested review from klcopp and lcspinter and removed request for abstractdog June 16, 2021 14:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants