Please sign in to comment.
Raft lock operation timeout fixes
Consider the following scenarios where client1 and client2 run on different threads: SCENARIO #1: - STEP 1: client1.lock(); Lock is acquired by client1. - STEP 2: client2.lock(); Wait key is added to the lock for client2. After some time, client2.lock() call fails with operation timeout - STEP3: client2.unlock(); Fails with IllegalMonitorStateException because client2 is not holder of the lock. However, its wait key is still present. So the lock will be assigned to client2 when client1 releases it. SCENARIO #2: - STEP 1: client1.lock(); Lock is acquired by client1. - STEP 2: client2.lock(); Wait key is added to the lock for client2. After some time, client2.lock() call fails with operation timeout - STEP 3: client2.lock(); A new wait key is added to the lock for client2, because its second lock() call has a new invocation uuid. - STEP 4: client1.unlock(); The lock will be assigned to client2 for its first lock() call. Now the lock is held by client2 and its lock count is just 1. However, nothing will be done for its other wait key in the list. - STEP 5: client2.unlock(); Client2 releases the lock, but since it has another wait key in the list, it will get the lock again. We need two separate fixes to resolve these issues. First, unlock() call should cancel all pending wait keys of the lock endpoint, if the lock is not currently held by itself. Second, if a lock() call is received while there is another wait key by the same client, the first lock() call must be cancelled via deleting its wait key from the wait list of the lock.
- Loading branch information...
Showing with 424 additions and 78 deletions.
- +3 −0 hazelcast-raft-client/src/main/java/com/hazelcast/raft/service/lock/client/RaftLockProxy.java
- +2 −0 ...ft-client/src/test/java/com/hazelcast/raft/service/lock/client/RaftFencedLockClientBasicTest.java
- +2 −0 ...ast-raft-client/src/test/java/com/hazelcast/raft/service/lock/client/RaftLockClientBasicTest.java
- +2 −0 ...st/java/com/hazelcast/raft/service/semaphore/client/RaftSessionAwareSemaphoreClientBasicTest.java
- +2 −0 ...est/java/com/hazelcast/raft/service/semaphore/client/RaftSessionlessSemaphoreClientBasicTest.java
- +3 −2 ...-raft-dataservices/src/main/java/com/hazelcast/raft/service/blocking/AbstractBlockingService.java
- +29 −17 hazelcast-raft-dataservices/src/main/java/com/hazelcast/raft/service/lock/LockRegistry.java
- +109 −30 hazelcast-raft-dataservices/src/main/java/com/hazelcast/raft/service/lock/RaftLock.java
- +65 −21 hazelcast-raft-dataservices/src/main/java/com/hazelcast/raft/service/lock/RaftLockService.java
- +51 −0 ...rvices/src/main/java/com/hazelcast/raft/service/lock/exception/LockRequestCancelledException.java
- +3 −0 ...dataservices/src/main/java/com/hazelcast/raft/service/lock/proxy/AbstractRaftFencedLockProxy.java
- +3 −0 hazelcast-raft-dataservices/src/main/java/com/hazelcast/raft/service/lock/proxy/RaftLockProxy.java
- +1 −2 ...cast-raft-dataservices/src/test/java/com/hazelcast/raft/service/lock/RaftFencedLockBasicTest.java
- +147 −0 hazelcast-raft-dataservices/src/test/java/com/hazelcast/raft/service/lock/RaftLockAdvancedTest.java
- +1 −3 hazelcast-raft-dataservices/src/test/java/com/hazelcast/raft/service/lock/RaftLockBasicTest.java
- +1 −3 ...rvices/src/test/java/com/hazelcast/raft/service/semaphore/RaftSessionAwareSemaphoreBasicTest.java
Oops, something went wrong.