New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Delete index folder if all shards were allocated away from a data only node #9985
Conversation
logger.debug("{} failed to delete unallocated shard, ignoring", ex, shardId); | ||
} | ||
// if the index doesn't exists anymore, delete its store as well, but only if its a non master node, since master | ||
// nodes keep the index metadata around | ||
if (indicesService.hasIndex(shardId.getIndex()) == false && currentState.nodes().localNode().masterNode() == false) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Just wondering why would master nodes want to keep the index directories around?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
for the index metadata
LGTM |
…ata only node If a folder for an index was created that folder is never deleted from that node unless the index is deleted. Data only nodes therefore can have empty folders for indices that they do not even have shards for. This commit makes sure empty folders are cleaned up after all shards have moved away from a data only node. The behavior is unchanged for master eligible nodes. closes #9985
} | ||
ClusterState clusterState = clusterService.state(); | ||
if (clusterState.metaData().hasIndex(indexName)) { | ||
if (clusterState.metaData().hasIndex(indexName) && (clusterState.nodes().localNode().masterNode() == true)) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
in this case isn't the exception wrong? I mean shouldn't we just skip it in that case?
If a folder for an index was created that folder is never deleted from that node unless the index is deleted.
Data only nodes therefore can have empty folders for indices that they do not even have shards for.
This commit makes sure empty folders are cleaned up after all shards have moved away from a data only
node. The behavior is unchanged for master eligible nodes.
I changed the
testShardActiveElseWhere
test to sometimes use data only nodes, not sure if this is useful. Also I added the check forclusterState.metaData().hasIndex(indexName)
inIndicesService.deleteIndexStore
again. Did not come up with a better way to solve this.