-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Teach tasks what machine they are running on #8190
Conversation
@Retention(RetentionPolicy.RUNTIME) | ||
@BindingAnnotation | ||
@PublicApi | ||
public @interface Parent |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
suggest ParentNode
instead to make it more clear what the annotation relates to
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm I want to make it consistent with Self
. Do you want me to rename it as well? I guess javadoc is pretty clear now that it relates to DruidNode
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hm, that's fine then, no need to rename in this patch
@jihoonson can you fix merge conflicts? |
@fjy fixed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
overall lgtm, but CliIndexer.java
needs to bind parent to self I think since #8107 has been merged.
@clintropolis thanks, fixed. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, thanks 👍
Part of #8061.
Description
In parallel native indexing, the middleManager is responsible for serving intermediary data for shuffle. As a result, the phase 2 tasks need to know where phase 1 tasks ran.
This PR lets tasks know where they are running by adding new system properties. This is supposed to be used for only internal purpose and so doesn't have to be documented.
This PR has: