Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reroute shards automatically when high disk watermark is exceeded #8270

Merged
merged 1 commit into from Oct 31, 2014

Conversation

dakrone
Copy link
Member

@dakrone dakrone commented Oct 29, 2014

This adds a Listener interface to the ClusterInfoService, this is used
by the DiskThresholdDecider, which adds a listener to check for nodes
passing the high watermark. If a node is past the high watermark an
empty reroute is issued so shards can be reallocated if desired.

A reroute will only be issued once every
cluster.routing.allocation.disk.reroute_interval, which is "60s" by
default.

Refactors InternalClusterInfoService to delegate the nodes stats and
indices stats gathering into separate methods so they have be overriden
by extending classes. Each stat gathering method returns a
CountDownLatch that can be used to wait until processing for that part
is successful before calling the listeners.

Fixes #8146

transportIndicesStatsAction.execute(indicesStatsRequest, new ActionListener<IndicesStatsResponse>() {
@Override
public void onResponse(IndicesStatsResponse response) {
listener.onResponse(response);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we count the latch down in a finally block?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we should have a LatchedActionListener that has this logic where we can just do new LatchedActionListener(delegate, latch)? I bet there or other places doing the same thing

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, I'll add a LatchedActionListener so this is useful for others

@dakrone
Copy link
Member Author

dakrone commented Oct 29, 2014

@s1monw pushed some commits addressing the feedback

* An action listener that allows passing in a {@link CountDownLatch} that
* will be counted down after onResponse or onFailure is called
*/
public class LatchedActionListener<T> implements ActionListener<T> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just being over paranoid I think this class should be final

@s1monw
Copy link
Contributor

s1monw commented Oct 30, 2014

left some minor comments - LGTM otherwise

@s1monw s1monw removed the review label Oct 30, 2014
@dakrone dakrone force-pushed the cluster-info-listener branch 2 times, most recently from a671bae to 4ac7b02 Compare October 31, 2014 10:58
This adds a Listener interface to the ClusterInfoService, this is used
by the DiskThresholdDecider, which adds a listener to check for nodes
passing the high watermark. If a node is past the high watermark an
empty reroute is issued so shards can be reallocated if desired.

A reroute will only be issued once every
`cluster.routing.allocation.disk.reroute_interval`, which is "60s" by
default.

Refactors InternalClusterInfoService to delegate the nodes stats and
indices stats gathering into separate methods so they have be overriden
by extending classes. Each stat gathering method returns a
CountDownLatch that can be used to wait until processing for that part
is successful before calling the listeners.

Fixes elastic#8146
@dakrone dakrone merged commit 4ac7b02 into elastic:master Oct 31, 2014
@dakrone dakrone deleted the cluster-info-listener branch November 11, 2014 12:47
@dakrone dakrone added the v1.4.1 label Nov 13, 2014
@clintongormley clintongormley changed the title Reroute shards automatically when high disk watermark is exceeded Allocation: Reroute shards automatically when high disk watermark is exceeded Nov 25, 2014
@clintongormley clintongormley changed the title Allocation: Reroute shards automatically when high disk watermark is exceeded Reroute shards automatically when high disk watermark is exceeded Jun 6, 2015
@lcawl lcawl added :Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. and removed :Allocation labels Feb 13, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed/Distributed A catch all label for anything in the Distributed Area. If you aren't sure, use this one. >enhancement v1.4.1 v1.5.0 v2.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Disk allocation not periodically checking for whether thresholds/watermarks are exceeded
4 participants