Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes a deadlock on internal executor when listeners are used #7944

Conversation

Projects
None yet
2 participants
@sancar
Copy link
Member

commented Apr 12, 2016

A similar deadlock was detected and fixed here
#7673

Both have the same reason. When remote invocation is waited
indefinetely on internal executor since response also same via
same executor and invocation monitor also runs on same executor,
there is change of deadlock.

As a fix, I have introduced a dedicated single thread executor to
be used by listener registration logic. This way, internal
executor will not be blocked.

I have introduced a property to make the test fail constantly.
"hazelcast.client.internal.executor.pool.size" . We should not
need to expose this to user right now. This is just for testing.

@sancar sancar self-assigned this Apr 12, 2016

@sancar sancar added this to the 3.6.3 milestone Apr 12, 2016

@@ -70,6 +70,8 @@ public boolean isDone() {

@Override
public ClientMessage get() throws InterruptedException, ExecutionException {
//Client should wait remote invocation indefinitely on internal thread

This comment has been minimized.

Copy link
@pveentjer

pveentjer Apr 12, 2016

Member

Smells like a hacky assert since you are relying on a threadname.

@sancar sancar force-pushed the sancar:fix/deadlockOnInternalExecutorWithListener/maint3.x branch 2 times, most recently from 0cc1834 to 30cb090 Apr 12, 2016

Fixes a deadlock on internal executor when listeners are used
A similar deadlock was detected and fixed here
#7673

Both have the same reason. When remote invocation is waited
indefinetely on internal executor since response also same via
same executor and invocation monitor also runs on same executor,
there is change of deadlock.

As a fix, I have introduced a dedicated single thread executor to
 be used by listener registration logic. This way, internal
 executor will not be blocked.

I have introduced a property to make the test fail constantly.
"hazelcast.client.internal.executor.pool.size" . We should not
need to expose this to user right now. This is just for testing.

@sancar sancar force-pushed the sancar:fix/deadlockOnInternalExecutorWithListener/maint3.x branch from 30cb090 to ec94026 Apr 18, 2016

@sancar sancar merged commit 937106d into hazelcast:maintenance-3.x Apr 19, 2016

1 check passed

default 10877 tests run, 39 skipped, 0 failed.
Details

@sancar sancar deleted the sancar:fix/deadlockOnInternalExecutorWithListener/maint3.x branch Apr 19, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.