-
Notifications
You must be signed in to change notification settings - Fork 1.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Improve][Connector-V2] Doris stream load use FE instead of BE #6235
Conversation
this.backends = RestService.getBackendsV2(dorisConfig, log); | ||
String backend = getAvailableBackend(); | ||
private void initializeLoad() { | ||
String backend = dorisConfig.getFrontends(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It looks like it just returns fe. I wonder what's the purpose here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is the purpose of this PR. Change stream load endpoint from BE to FE.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fe is mainly responsible for doris's query plan. I am worried that if stream load also selects fe, the efficiency of fe will be affected in high concurrency scenarios.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for your response.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
+1
Purpose of this pull request
This PR change doris stream load endpoint from BE to FE. Before this PR, we randomly selected BE, and it would be better to let Doris FE judge the ability to select BE.
Does this PR introduce any user-facing change?
no
How was this patch tested?
exist test.
Check list
New License Guide
release-note
.