Skip to content

[fix](coordinator) Fix wrong recvrId in fragment contains BHJ (#47727)#47729

Merged
Gabriel39 merged 1 commit intoapache:branch-2.1from
Gabriel39:pick_0211_2
Feb 11, 2025
Merged

[fix](coordinator) Fix wrong recvrId in fragment contains BHJ (#47727)#47729
Gabriel39 merged 1 commit intoapache:branch-2.1from
Gabriel39:pick_0211_2

Conversation

@Gabriel39
Copy link
Contributor

In Coordinator, a shuffle map consists of recvrId in each instance. For example, if 3 BEs exist in a cluster, for a shuffled hash join, we get 3 maps for a fragment sent to each BE:
BE0: {0:0, 1:1}
BE1: {2:0, 3:1}
BE2: {4:0, 5:1}
In this example, parallelism is 2. Keys in shuffle map indicate the global shuffle id and the values indicate the instance id in current BE. In Coordinator, the recvrId is the global shuffle id of each instance so we may get a wrong result if it is wrong.

This bug is caused by recvrId set by a BHJ fragment. If a fragment contains both BHJ and SHJ, recvrId should be set by SHJ and BHJ should be ignored.

What problem does this PR solve?

Issue Number: close #xxx

Related PR: #xxx

Problem Summary:

Release note

None

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

…he#47727)

In Coordinator, a shuffle map consists of `recvrId` in each instance.
For example, if 3 BEs exist in a cluster, for a shuffled hash join, we
get 3 maps for a fragment sent to each BE:
BE0: {0:0, 1:1}
BE1: {2:0, 3:1}
BE2: {4:0, 5:1}
In this example, parallelism is 2. Keys in shuffle map indicate the
global shuffle id and the values indicate the instance id in current BE.
In Coordinator, the `recvrId` is the global shuffle id of each instance
so we may get a wrong result if it is wrong.

This bug is caused by `recvrId` set by a BHJ fragment. If a fragment
contains both BHJ and SHJ, `recvrId` should be set by SHJ and BHJ should
be ignored.
@Gabriel39
Copy link
Contributor Author

run buildall

@Thearas
Copy link
Contributor

Thearas commented Feb 11, 2025

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.50% (9610/26327)
Line Coverage: 28.08% (79641/283582)
Region Coverage: 26.74% (40851/152744)
Branch Coverage: 23.51% (20711/88110)
Coverage Report: http://coverage.selectdb-in.cc/coverage/719d963dbd42b2517a648dfacd34fcef796d5c93_719d963dbd42b2517a648dfacd34fcef796d5c93/report/index.html

@Gabriel39 Gabriel39 merged commit 964c940 into apache:branch-2.1 Feb 11, 2025
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants