…ache has finished starting up
…d the prepare command, since they could have already left the cluster. JGroups will wait for the response message until the timeout replication expires, while holding the locks for the modified keys. So other transactions could time out waiting for our locks.
…HandlerImpl if the entire cache manager is stopping
…ions or not
…o start properly This allowed an inbound command to start before the initial rehash even started.
…logger lock even if the latch was open Sometimes the rehasher can acquire the write lock after a transaction thread has called latch.await() but before readLock.lock(). If that happens we need to retry the lock acquisition in the transaction thread.
This is no longer strictly necessary for ISPN-1106, as we are waiting with a shorter timeout on transactions with locks and so the rehash does not block for a very long period of time. It is recommended however to start all caches on application startup, and this method provides an easy way for users to start all their caches.
…hash from finishing The generic scenario involves multiple caches. Say we have transactions Tx1 and Tx2 spanning caches C1 and C2. A new node joins the cluster, starting C1 and C2. With the following sequence of events rehashing will be blocked for lockAcquisitionTimeout. 1. Tx1 prepares on C1 locking K1 2. Tx2 wants to prepare on C2, Tx2 gets the tx lock 3. Tx2 now waits to lock K1 while holding the tx lock on C2 4. Rehash starts on C2 but it can't proceed because Tx2 has the tx lock 5. Tx1 now wants to prepare on C2, but can't acquire the tx lock I've implemented a crude "deadlock detection" scheme: a new tx will wait the full lockAcquisitionTimeout for the tx lock, but a tx that already has locks acquired will only wait 1/100 of that. So if there is a cycle it will break much quicker and allow rehashing to proceed. There is also a simpler variant where the transactions work with a single cache. In that case if the remote command can't acquire the tx lock with 0 timeout it knows that it has the tx lock on the origin node and it's in a deadlock situation.
…o the TopologyChanged event.
…nceTask can invalidate the keys after rehashing is done but before the cache listeners (e.g. KeyAffinityService) know it.
* In LockManagerImpl log the other keys owned by the current transaction. * In DefaultCacheManager push the cache name to the NDC during cache startup. * Improved toString() for RehashControlCommand and DistributedExecuteCommand. * In InboundInvocationHandler log the cache name. * Log cache start/stop. * Log the read lock owners in JGroupsDistSync.
…y were waiting for the default cache to finish rehashing.
…re and after rehash can be disjoint
…mote nodes If a rehash started on a remote node, and we're already holding the lock on the origin there's a potential for deadlock. So we unlock the tx lock to allow rehash to progress on the local node and retry all the remote calls.
…out to kick in
…sure that we also got the updated views