Expand ClusterInfo to provide min / max disk usage for allocation decider #13163
Conversation
| @@ -77,7 +83,7 @@ public long getUsedBytes() { | |||
|
|
|||
| @Override | |||
| public String toString() { | |||
| return "[" + nodeId + "][" + nodeName + "] free: " + new ByteSizeValue(getFreeBytes()) + | |||
| return "[" + nodeId + "][" + nodeName + "][" + path + "] free: " + new ByteSizeValue(getFreeBytes()) + | |||
|
LGTM, but worth adding something mentioning it in the documentation? |
@dakrone you mean the main documenation or javadocs? |
|
The main documentation is mostly what I meant (javadocs never hurt though! ;) ) |
|
honestly I don't know what I should add there... I am just fixing a bug here not adding functionality? |
Maybe something like this: "Prior to 2.0.0, when using multiple data paths, the disk threshold I think it's mostly for people upgrading from 1.x to 2.x to understand a |
|
It's optional though, it doesn't need to block this PR going in |
|
@dakrone yeah I am still trying to digest if people need to know about this impl detail? |
|
Usually I would say no, but because this can make the difference between "why were my shards allocated in 1.7 but not in 2.0?" I think it is a useful thing to add |
s1monw
force-pushed
the
min_max_disk_space
branch
from
7d7c062
to
60b12ec
Compare
dakrone
changed the title
I will make sure I add this before I close #13106 |
Sounds good to me! Thanks! |
s1monw
force-pushed
the
min_max_disk_space
branch
from
60b12ec
to
c05e9ba
Compare
…cider Today we sum up the disk usage for the allocation decider which is broken since we don't stripe across multiple data paths. Each shard has it's own private path now but the allocation deciders still treat all paths as one big disk. This commit adds allows allocation deciders to access the least used and most used path to make better allocation decidsions upon canRemain and canAllocate calls. Yet, this commit doesn't fix all the issues since we still can't tell which shard can remain and which can't. This problem is out of scope in this commit and will be solved in a followup commit. Relates to elastic#13106
s1monw
force-pushed
the
min_max_disk_space
branch
from
c05e9ba
to
0c71328
Compare
|
@s1monw i assume you're still planning on backporting this to 2.0? I fixed the version label for master. |
lcawl
added
:Distributed/Distributed
clintongormley
added
:Distributed/Allocation
Today we sum up the disk usage for the allocation decider which is broken since
we don't stripe across multiple data paths. Each shard has it's own private path
now but the allocation deciders still treat all paths as one big disk. This commit
adds allows allocation deciders to access the least used and most used path to make
better allocation decisions upon canRemain and canAllocate calls.
Yet, this commit doesn't fix all the issues since we still can't tell which shard
can remain and which can't. This problem is out of scope in this commit and will be solved
in a followup commit.
Relates to #13106