[Improve Doc] : Modify the disadvantages of the lazyLoadOnStart feature.#10608
Merged
jihoonson merged 2 commits intoapache:masterfrom Dec 2, 2020
Merged
[Improve Doc] : Modify the disadvantages of the lazyLoadOnStart feature.#10608jihoonson merged 2 commits intoapache:masterfrom
jihoonson merged 2 commits intoapache:masterfrom
Conversation
added 2 commits
November 26, 2020 17:49
Contributor
Author
|
I will keep digging and try to make an another PR to fix this problem of lazyOnStart. |
capistrant
approved these changes
Nov 27, 2020
jihoonson
approved these changes
Dec 2, 2020
Contributor
Author
|
@jihoonson @nishantmonu51 @capistrant Thanks for your review and merge! |
Contributor
Author
I just make a PR(#10650) to solve the problem described in the quote and tested on Druid Cluster. |
9 tasks
Contributor
|
@zhangyue19921010 I guess this doc should be modified again according to #10688 . The corrupted segment doesn't need manual deletion now. |
Contributor
Author
|
Hi @kaijianding. Thanks for reminding. I will make a further PR to modified docs again today. |
Contributor
Author
|
done. #10778 |
JulianJaffePinterest
pushed a commit
to JulianJaffePinterest/druid
that referenced
this pull request
Jan 22, 2021
…re. (apache#10608) * modify docs * modify docs Co-authored-by: yuezhang <yuezhang@freewheel.tv>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Conclusion
meta.smooshmaliciously, Historical node with lazyOnStart enable will startup successfully but queries using these segments would fail at runtime.Here is the work flow of historical server loading segments.
Condition 1: When Historical Server re-start, it will try to load local cached segments listed in
info_dir.Condition 2: Historical will create a specific file named by a corresponding segmentID in
info_dirafter downloading/creating segment files successfully.If Historical crashed in the process of downloading/creating segment files before
info_dir filecreated, Historical would not try to load this segments when startup lazily. And the loading action of this segment will be triggered by ZKCurator which is a unLazy action.Un-lazy loading action will throw exception because of damaged segment files.
And will delete all the damaged files.
If Historical crashed after
info_dir filecreated, it means all the segments file is complete.What's more!
If served segments is damaged like manually damage
meta.smooshmaliciously, Historical node with lazyOnStart enable will startup successfully but queries using these segments would fail at runtime.I delete
video_cro_network_name,0,23837261,23910951inmeta.smooshmanually and restart the historical node(enable lazyOnStart).Historical server startup successfully but queries is failed.
This PR has:
Key changed/added classes in this PR
index.md