fix(plan): avoid shuffle for replace capture dedup join - #24502
Merged
Conversation
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
aunjgr
approved these changes
May 20, 2026
aunjgr
left a comment
aunjgr
left a comment
Contributor
There was a problem hiding this comment.
Review: disable shuffle for OldColCapture dedup joins
Approve.
No blocking issues found. The change correctly prevents DEDUP joins with OldColCaptureList from being planned as shuffle joins, avoiding the unsupported compile path that would panic for shuffle OldColCapture dedup joins.
The added tests cover both the generic dedup shuffle decision and the actual REPLACE capture-plan shape with large build-side stats.
Contributor
Merge Queue Status
This pull request spent 1 hour 2 minutes 13 seconds in the queue, including 1 hour 58 seconds running CI. Required conditions to merge
|
7 tasks
mergify Bot
pushed a commit
that referenced
this pull request
May 25, 2026
…to 4.0-dev) (#24518) Cherry-pick #24502 to `4.0-dev`. `REPLACE INTO ... SELECT ... FROM` can build a DEDUP join with `OldColCaptureList` on the merged main-table scan path. For large source scans, the planner could still mark that DEDUP join as shuffle, but the compile layer does not support shuffle DEDUP joins with old-column capture and fails with: ```text shuffle DedupJoin with OldColCapture is not supported ``` This PR disables shuffle planning for DEDUP joins that carry `DedupJoinCtx.OldColCaptureList`, keeping those REPLACE plans on the supported non-shuffle capture path. The existing `OldColList` guard is preserved. Cherry-picked commit: - 66f9089 `fix(plan): avoid shuffle for replace capture dedup join` Validation: - Not run locally for this cherry-pick PR. Approved by: @aunjgr, @XuPeng-SH
ULookup
pushed a commit
to ULookup/matrixone
that referenced
this pull request
May 26, 2026
…#24502) `REPLACE INTO ... SELECT ... FROM` can build a DEDUP join with `OldColCaptureList` on the merged main-table scan path. For large source scans, the planner could still mark that DEDUP join as shuffle, but the compile layer does not support shuffle DEDUP joins with old-column capture and fails with: ```text shuffle DedupJoin with OldColCapture is not supported ``` This PR disables shuffle planning for DEDUP joins that carry `DedupJoinCtx.OldColCaptureList`, keeping those REPLACE plans on the supported non-shuffle capture path. The existing `OldColList` guard is preserved. Tests added: - cover plain large DEDUP joins still being eligible for shuffle - cover `OldColList` and `OldColCaptureList` disabling shuffle - cover the actual REPLACE merged-scan plan shape so capture DEDUP joins do not become shuffle plans Validation: - `go test ./pkg/sql/plan -run 'TestReplaceCaptureList|TestReplaceCaptureDedupJoinDoesNotShuffle|TestDetermineShuffleForDedupJoin' -count=1`\n- `go test ./pkg/sql/plan -count=1`\n- `git diff --check`\n- `make`\n- `make static-check` Approved by: @aunjgr
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
Which issue(s) this PR fixes:
issue #24472
What this PR does / why we need it:
REPLACE INTO ... SELECT ... FROMcan build a DEDUP join withOldColCaptureListon the merged main-table scan path. For large source scans, the planner could still mark that DEDUP join as shuffle, but the compile layer does not support shuffle DEDUP joins with old-column capture and fails with:This PR disables shuffle planning for DEDUP joins that carry
DedupJoinCtx.OldColCaptureList, keeping those REPLACE plans on the supported non-shuffle capture path. The existingOldColListguard is preserved.Tests added:
OldColListandOldColCaptureListdisabling shuffleValidation:
go test ./pkg/sql/plan -run 'TestReplaceCaptureList|TestReplaceCaptureDedupJoinDoesNotShuffle|TestDetermineShuffleForDedupJoin' -count=1\n-go test ./pkg/sql/plan -count=1\n-git diff --check\n-make\n-make static-check