Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

DiskThresholdDecider adds shard size even if shard is on different filesystem #17460

Closed
dakrone opened this issue Mar 31, 2016 · 0 comments
Closed

DiskThresholdDecider adds shard size even if shard is on different filesystem #17460

dakrone opened this issue Mar 31, 2016 · 0 comments
Assignees
@dakrone
Labels
>bug :Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes)

Comments

@dakrone
Copy link
Member

dakrone commented Mar 31, 2016

If the DiskThresholdDecider is enabled, it will attempt to see how much disk
space is used after a shard has been allocated to a node (to know whether it
is over the high watermark).

However, in the case that a shard is using a custom data_path setting, the
shard's size may not affect the amount of disk on the node's configured
path.data.

This can lead to weird things like shadow replicas not being relocated within a
cluster because all nodes think they don't have enough space for them, even
though they do because it's a different filesystem entirely.
@dakrone dakrone self-assigned this Mar 31, 2016
@dakrone dakrone added the >bug label Mar 31, 2016
dakrone added a commit to dakrone/elasticsearch that referenced this issue Apr 4, 2016
@dakrone
When considering the size of shadow replica indices, set size to 0
8e01b09 
Otherwise, when trying to calculate the amount of disk usage *after* the
shard has been allocated, it has incorrectly subtracted the shadow
replica size.

Resolves elastic#17460
@dakrone dakrone mentioned this issue Apr 4, 2016
Merged
dakrone added a commit that referenced this issue Apr 7, 2016
@dakrone
When considering the size of shadow replica indices, set size to 0
e334e83 
Otherwise, when trying to calculate the amount of disk usage *after* the
shard has been allocated, it has incorrectly subtracted the shadow
replica size.

Resolves #17460
dakrone added a commit that referenced this issue Apr 7, 2016
@dakrone
When considering the size of shadow replica indices, set size to 0
43f939c 
Otherwise, when trying to calculate the amount of disk usage *after* the
shard has been allocated, it has incorrectly subtracted the shadow
replica size.

Resolves #17460
@lcawl lcawl added :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. and removed :Allocation labels Feb 13, 2018
@clintongormley clintongormley added :Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes) and removed :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. labels Feb 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dakrone dakrone

Labels
>bug :Distributed/Allocation All issues relating to the decision making around placing a shard (both master logic & on the nodes)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dakrone @clintongormley @lcawl