Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Do not retry TargetNotMemberException when invocation made on target #10405

Merged
merged 1 commit into from Apr 28, 2017

Conversation

sancar
Copy link
Contributor

@sancar sancar commented Apr 20, 2017

Invocation made ProxyManager.initialize(ClientProxy) was on connection
to be able to prevent it from retrying. It was a workaround that does not
comply with ClientInvocation API hence failed today.

In this pr, I made the proxy creation invocation on address as intended.
And I changed invocation exception handling so that it will not be retried
when given address not a member anymore. Retrying does not make sense in
this case.

ClientLockWithTerminationTest.testLockOnClient_withNodeCrash was failing
because it was retrying creating proxy on closed member until
clientInvocationTimeout(120 seconds) hit and then throws OperationTimeoutException.

This pr #10370 made issue apparent.
It was waiting for 120 seconds unncessarily, then throwing a
retryableException(instead OperationTimeoutException). Because of retyable
exception there issue was hidden.

fixes #3422

@sancar sancar added this to the 3.9 milestone Apr 20, 2017
@sancar sancar self-assigned this Apr 20, 2017
@devOpsHazelcast
Copy link
Collaborator

@sancar
Copy link
Contributor Author

sancar commented Apr 20, 2017

run-lab-run

@devOpsHazelcast
Copy link
Collaborator

Test PASSed.

@@ -182,6 +184,11 @@ public void notifyException(Throwable exception) {
return;
}

if (address != null && exception instanceof TargetNotMemberException) {
Copy link
Contributor

@ihsandemir ihsandemir Apr 24, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will prevent all invocations which receive TargetNotMemberException being not retried even when it is actually retryable and it was being retried before. E.g. IMap.get() will end up at invocationService.invokeOnTarget() where address != null. Don't we want to retry these invocations?

Copy link
Contributor Author

@sancar sancar Apr 25, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My thinking was as following:
TargetNotMemberException is thrown when address is not in memberlist. It can be thrown from either server or client.
If invocation intended to be send to an address (address != null), it will retry always in same address. If address is not in memberlist, there is no point retrying. It does not matter if client message is retyable or not it will never be executed.
Makes sense ?

Note: IMap.get does not use invokeOnTarget, it uses invokeOnPartition

@@ -171,6 +172,7 @@ public void notify(ClientMessage clientMessage) {
clientInvocationFuture.complete(clientMessage);
}

@SuppressWarnings({ "checkstyle:cyclomaticcomplexity", "checkstyle:npathcomplexity" })
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there no workaround not to introduce this suppression?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The workaround would be to simplify the method, so less branches etc. (normally you can achieve this by pulling out methods).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not in favor of pulling out methods when methods are not reused. But I have a pr cookinf up built on top of this. It will simplify these parts by introducing invocation classes for different targets. There I will remove back the suppressions.

@sancar sancar force-pushed the fix/targetNotMember/master branch from b6e688b to 436446a Compare April 26, 2017 07:35
@devOpsHazelcast
Copy link
Collaborator

Test PASSed.

@sancar sancar force-pushed the fix/targetNotMember/master branch from 436446a to 8c8e1c5 Compare April 26, 2017 12:10
Invocation made `ProxyManager.initialize(ClientProxy)` was on connection
to be able to prevent it from retrying. It was a workaround that does not
comply with ClientInvocation API hence failed today.

In this pr, I made the proxy creation invocation on address as intended.
And I changed invocation exception handling so that it will not be retried
when given address not a member anymore. Retrying does not make sense in
this case.

ClientLockWithTerminationTest.testLockOnClient_withNodeCrash was failing
because it was retrying creating proxy on closed member until
clientInvocationTimeout(120 seconds) hit and then throws OperationTimeoutException.

This pr hazelcast#10370 made issue apparent.
It was waiting for 120 seconds unncessarily, then throwing a
retryableException(instead OperationTimeoutException). Because of retyable
exception there issue was hidden.

fixes hazelcast#3422
@devOpsHazelcast
Copy link
Collaborator

Test PASSed.

@sancar sancar merged commit 097da35 into hazelcast:master Apr 28, 2017
@sancar sancar deleted the fix/targetNotMember/master branch April 28, 2017 07:23
@mmedenjak mmedenjak added the Source: Internal PR or issue was opened by an employee label Apr 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Source: Internal PR or issue was opened by an employee Team: Client Type: Defect
Projects
None yet
Development

Successfully merging this pull request may close these issues.

ClientLockWithTerminationTest.testLockOnClient_withNodeCrash
6 participants