Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cancel recovery if shard on the target node closes during recovery operation #6645

Merged

Conversation

Projects
None yet
4 participants
@martijnvg
Copy link
Member

commented Jun 30, 2014

On the target side if the shard closes while recovery is in progress (e.g. files being transferred), the recovery operation should be cancelled.

@bleskes

This comment has been minimized.

Copy link
Member

commented Jun 30, 2014

LGTM - I would change the title of the PR to indicate this is about closing the shard on the target node. I was thinking how we can test it. Maybe a test that issues a relocation command, then overrides it with an allocation filtering rule. This will at least make sure the code kicks in every once in a while

@martijnvg martijnvg changed the title Cancel recovery if shard closes during recovery operation Cancel recovery if shard on the target node closes during recovery operation Jun 30, 2014

@martijnvg

This comment has been minimized.

Copy link
Member Author

commented Jun 30, 2014

@bleskes I added a test that is likely to cancel an ongoing recovery.

@bleskes

This comment has been minimized.

Copy link
Member

commented Jun 30, 2014

@martijnvg thx. Left some comments on the commit (sorry, notice too late it's not the PR)

@bleskes

View changes

src/test/java/org/elasticsearch/recovery/RelocationTests.java Outdated

ClusterStateResponse stateResponse = client().admin().cluster().prepareState().get();
String blueNodeId = null;
for (DiscoveryNode discoveryNode : stateResponse.getState().nodes()) {

This comment has been minimized.

Copy link
@bleskes

bleskes Jul 1, 2014

Member

Nit picky: if we capture the node name from the start async we can do internalCluster().getInstance(DiscoveryNode.class, blueNodeName).id()

).get();

// Lets wait a bit and then move again to hopefully trigger recovery cancellations.
boolean applied = awaitBusy(

This comment has been minimized.

Copy link
@bleskes

bleskes Jul 1, 2014

Member

I think this may miss a recovery if it is completed already. I think you should look for a recovery on the red node and not care if it's active or not. Also, maybe slow down the recovery as well? see - https://github.com/elasticsearch/elasticsearch/blob/c907ce325e89b25d0ff29288f018adda35bbdeed/src/test/java/org/elasticsearch/indices/recovery/IndexRecoveryTests.java#L104

@martijnvg

This comment has been minimized.

Copy link
Member Author

commented Jul 1, 2014

@bleskes Good points, I applied the feedback.

@bleskes

This comment has been minimized.

Copy link
Member

commented Jul 1, 2014

LGTM!

martijnvg added a commit that referenced this pull request Jul 1, 2014

@martijnvg martijnvg merged commit 5668b1c into elastic:master Jul 1, 2014

@martijnvg martijnvg changed the title Cancel recovery if shard on the target node closes during recovery operation Recovery: Cancel recovery if shard on the target node closes during recovery operation Jul 2, 2014

@magnhaug

This comment has been minimized.

Copy link

commented Jul 5, 2014

Does this solve issue #6430 ?

@clintongormley clintongormley changed the title Recovery: Cancel recovery if shard on the target node closes during recovery operation Resiliency: Cancel recovery if shard on the target node closes during recovery operation Jul 16, 2014

@martijnvg martijnvg deleted the martijnvg:improvements/recovery_cancel_if_closed branch May 18, 2015

@clintongormley clintongormley changed the title Resiliency: Cancel recovery if shard on the target node closes during recovery operation Cancel recovery if shard on the target node closes during recovery operation Jun 7, 2015

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.