[fix](be) Continue sorted merge when sender queue is ready#65004
Merged
zclllyybb merged 1 commit intoJul 1, 2026
Merged
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
/review |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Review completed for PR 65004. I did not find a blocking correctness issue or any inline comment worth submitting.
Critical checkpoint conclusions:
- Goal/test proof: the PR keeps sorted exchange merging from yielding early when the exhausted sender queue already has another block or EOS ready, while preserving the pending-cursor path when it is not ready. The added BE unit tests cover eager-ready continuation, not-ready pending behavior, string-column row materialization after mid-loop flushing, and existing offset/limit/single-stream cases.
- Scope/focus: the code change is small and confined to the stream receiver ready check, sorted cursor callback, merger loop, and targeted BE unit tests.
- Concurrency/lifecycle: sender queue readiness is read under the same mutex that protects queue/cancel/EOS transitions; no new thread, lock order, or object lifetime owner is introduced.
- Configuration/compatibility: no new config/session variable, FE-BE protocol field, storage format, or rolling-upgrade compatibility surface is introduced.
- Parallel paths: non-exchange VSortedRunMerger callers keep old behavior through the default empty ready checker; sorted exchange receivers pass the new checker explicitly.
- Testing/validation: scoped git diff --check passed. I could not run the BE UT in this checkout because thirdparty/installed is absent and no prebuilt sort_merger_test binary exists.
User focus: no additional user-provided review focus was present.
Subagent conclusions: optimizer-rewrite and tests-session-config both reported no candidates in the initial pass. No candidates were accepted, dismissed as duplicates, or turned into inline comments. Convergence round 1 ended with both live subagents replying NO_NEW_VALUABLE_FINDINGS for this same summary-only final comment set.
Contributor
Author
|
run buildall |
Contributor
TPC-H: Total hot run time: 29954 ms |
Contributor
TPC-DS: Total hot run time: 174427 ms |
Contributor
ClickBench: Total hot run time: 25.17 s |
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
zclllyybb
approved these changes
Jul 1, 2026
github-actions Bot
pushed a commit
that referenced
this pull request
Jul 1, 2026
### What problem does this PR solve? When a sorted-run merge cursor reaches the end of its current block and the sender already has the next block ready, the merger may fetch the next block before flushing rows that have already been selected into the output block. For variable-length columns, the pending row addresses still point to the previous cursor block. Reusing the cursor block before `do_insert()` can make the output read rows from the wrong block and corrupt string offsets. ### What is changed? Flush pending output rows before loading the next ready block from the exhausted cursor. This keeps the existing non-ready path behavior unchanged: when the next block is not ready, the cursor is saved as pending and the current output block is returned first. The PR also adds regression coverage for both: - continuing merge when the next sender block is already ready - preserving string column values before the cursor block is reused
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
When a sorted-run merge cursor reaches the end of its current block and the sender already has the next block ready, the merger may fetch the next block before flushing rows that have already been selected into the output block.
For variable-length columns, the pending row addresses still point to the previous cursor block. Reusing the cursor block before
do_insert()can make the output read rows from the wrong block and corrupt string offsets.What is changed?
Flush pending output rows before loading the next ready block from the exhausted cursor.
This keeps the existing non-ready path behavior unchanged: when the next block is not ready, the cursor is saved as pending and the current output block is returned first.
The PR also adds regression coverage for both:
Check List
./run-be-ut.sh -j 48 --run --filter=SortMergerTest.*