HIVE-27480: OFFSET without ORDER BY generates wrong results #4511
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
kgyrtkirk
added
tests pending
tests unstable
tests passed
and removed
tests pending
tests unstable
labels
|
Kudos, SonarCloud Quality Gate passed!
|
okumin
changed the title
okumin
commented
Jul 24, 2023
| @@ -1099,7 +1116,7 @@ limit 1 offset 1 | |||
| POSTHOOK: type: QUERY | |||
| POSTHOOK: Input: default@src | |||
| #### A masked pattern was here #### | |||
| 86 val_86 86 val_86 | |||
| 238 val_238 238 val_238 | |||
There was a problem hiding this comment.
This is OK because the test query picks up any single row.
select *
from src src1 left outer join src src2
on src1.key = src2.key
limit 1 offset 1
|
@kasakrisz Could you please take a look when you have a chance? This PR is related to #4471. |
kasakrisz
reviewed
Jul 26, 2023
asf-ci-hive
added
tests failed
tests pending
tests unstable
and removed
tests pending
tests failed
tests unstable
labels
asf-ci-hive
added
tests pending
tests failed
and removed
tests pending
tests failed
labels
|
Kudos, SonarCloud Quality Gate passed!
|
kasakrisz
approved these changes
Sep 8, 2023
|
Thanks for your review! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Add an additional shuffle with a single reducer when OFFSET is used without ORDER BY.
Why are the changes needed?
Prevent data integrity issues.
https://issues.apache.org/jira/browse/HIVE-27480
Does this PR introduce any user-facing change?
Execution plans can change but it should be OK since the original one had risks of wrong results.
This PR would also add a new param but the default value wouldn't change the behavior.
Is the change a dependency upgrade?
Not
How was this patch tested?
I added and updated itests