Add node-level/Disk-level merge rate throttling #51140
Labels
:Distributed/Engine
Anything around managing Lucene and the Translog in an open shard.
Team:Distributed
Meta label for distributed team
ES_VERSION: 7.3.1
Description of the problem including actual behavior:
We know that ES will throttle the merge rate with shard-level, it concerns the two factors:
It has good control over the rate merge in a shard, but a node will have many shards and the merges rate among the shards are independently, It's very likely that two shards of one node belonging to one disk are undergoing merging at high IO usage, the io usage of the disk will be 100%, then the node will become write bottleneck, the performance of the cluster will be poor, It happens many times in product.
expected behavior:
If we could also throttle IO usage at node-level, not just shard-level when the segments are merging? or if we could throttle IO usage at disk-level?
The text was updated successfully, but these errors were encountered: