New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Mapping update task back references already closed index shard #9607
Conversation
I found this due to heapdump analysis on a test OOM happening 2 days ago on the |
@@ -659,4 +618,72 @@ public String toString() { | |||
", targetNode=" + request.targetNode() + | |||
'}'; | |||
} | |||
|
|||
// this is a static class since we are holding and instance to the IndexShard |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"and instance to" -> "an instance of"
LGTM |
1 similar comment
LGTM |
|
||
@Override | ||
public ClusterState execute(ClusterState currentState) throws Exception { | ||
if (cancellableThreads.isCancelled() == false) { // no need to run this if recovery is canceled |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nice extra touch
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah I thought we don't need to do the extra work if the queue is already long :)
LGTM as well. Good catch. |
In the ShardRecoveryHandler we issue cluster update tasks to update the mapping. The annonymous inner class backreferences the ShardRecoveryHandler which holds a potentially large IndexShard object (which references buffers & caches etc) If the queue of update tasks piles up and recoveries get cancled and/or shards are closed the ShardRecoveryHandler can't be GCed. This commit moves the update task into a static inner class to allos the GC to do its job.
b4c532e
to
b3b1a11
Compare
In the ShardRecoveryHandler we issue cluster update tasks to update the
mapping. The annonymous inner class backreferences the ShardRecoveryHandler
which holds a potentially large IndexShard object (which references buffers & caches etc)
If the queue of update tasks piles up and recoveries get cancled and/or shards are closed
the ShardRecoveryHandler can't be GCed. This commit moves the update task into a static
inner class to allos the GC to do its job.