Broker: Await initialization before finishing startup.#6742
Merged
gianm merged 3 commits intoapache:masterfrom Dec 19, 2018
Merged
Broker: Await initialization before finishing startup.#6742gianm merged 3 commits intoapache:masterfrom
gianm merged 3 commits intoapache:masterfrom
Conversation
In particular, hold off on announcing the service and starting the HTTP server until the server view and SQL metadata cache are finished initializing. This closes a window of time where a Broker could return partial results shortly after startup. As part of this, some simplification of server-lifecycle service announcements. This helps ensure that the two different kinds of announcements we do (legacy and new-style) stay in sync.
Member
clintropolis
left a comment
clintropolis
left a comment
There was a problem hiding this comment.
👍 after CI (failing currently due to unused imports)
gianm
added a commit
to implydata/druid-public
that referenced
this pull request
Dec 19, 2018
* Broker: Await initialization before finishing startup. In particular, hold off on announcing the service and starting the HTTP server until the server view and SQL metadata cache are finished initializing. This closes a window of time where a Broker could return partial results shortly after startup. As part of this, some simplification of server-lifecycle service announcements. This helps ensure that the two different kinds of announcements we do (legacy and new-style) stay in sync. * Remove unused imports. * Fix NPE in ServerRunnable.
gianm
added a commit
to gianm/druid
that referenced
this pull request
Dec 20, 2018
Fixes a bug introduced in apache#6742, where the broker would delay startup indefinitely if there were no segments at all being served by any data servers.
dclim
pushed a commit
to implydata/druid-public
that referenced
this pull request
Dec 20, 2018
…pache#6765) Fixes a bug introduced in apache#6742, where the broker would delay startup indefinitely if there were no segments at all being served by any data servers.
This was referenced Jan 11, 2019
Contributor
Author
|
Should also help work around #6176 until we can figure out why that's happening (the latest thought is that it is something Curator related?). The broker would stay stuck waiting for startup forever, which would at least mean it won't run with a broken SQL metadata cache. The issue would be fully worked-around if paired with monitoring that restarts unhealthy brokers. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
In particular, hold off on announcing the service and starting the
HTTP server until the server view and SQL metadata cache are finished
initializing. This closes a window of time where a Broker could return
partial results shortly after startup.
I decided to do it this way because there are lots of ways people find
Brokers to query, including:
Delaying startup should play well with all three mechanisms.
As part of this, some simplification of server-lifecycle service
announcements. This helps ensure that the two different kinds of
announcements we do (legacy and new-style) stay in sync.