Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ISPN-3048 Eviction needs to be transactional #2252

Closed
wants to merge 1 commit into from

Conversation

pruivo
Copy link
Member

@pruivo pruivo commented Dec 2, 2013

  • Fixed issues with manual eviction to make it more reliable
  • Added tests for manual and size based eviction which happens
    concurrently with other operations

Notes:

  • The manual eviction sends the eviction command to the primary owner to lock the key and sends back the command to the originator to remove the key from in-memory data container.
  • Also, in ActivateInterceptor, the key is locked and it double checks the data container before load and remove from persistence
  • This is a patch for now. New implementation will be integrated in 7.0 (https://issues.jboss.org/browse/ISPN-3797)

https://issues.jboss.org/browse/ISPN-3048

return controlledPassivationManager;
}

protected final Future<Object> putWithFuture(final Object key, final Object value) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, why not call putAsync?

@galderz
Copy link
Member

galderz commented Dec 10, 2013

@pruivo Not sure if you noticed, but I previously created org.infinispan.eviction.ConcurrentPassivationActivationTest class which tests some scenarios. There should be some consolidation so that these tests are either merged with your tests, or they are removed if they're superseded by your tests. Great work btw :)

@pruivo
Copy link
Member Author

pruivo commented Dec 10, 2013

@galderz thanks for the heads up. I take a look to ConcurrentPassivationActivationTest and it looks very similar to my scenario 2 (eviction blocked after passivation and concurrent get operation). Can I remove it?

@galderz
Copy link
Member

galderz commented Dec 10, 2013

Sure, you can remove it.

@pruivo
Copy link
Member Author

pruivo commented Dec 10, 2013

updated!

@mmarkus
Copy link
Contributor

mmarkus commented Dec 12, 2013

pulling..

@mmarkus
Copy link
Contributor

mmarkus commented Dec 12, 2013

EvictionWithPassivationAndConcurrentOperationsTest>EvictionWithConcurrentOperationsTest.testScenario6:352->assertInMemory:44 Wrong value for key key1 in data container expected: but was:

@pruivo
Copy link
Member Author

pruivo commented Dec 12, 2013

@mmarkus do you have any logs about it? I cannot reproduce it :(

cdl.commitEntry(e, null, cmd, ctx);
removeFromStoreIfNeeded(key);
try {
cdl.lock(key);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch if 2 threads come in and 1 activates before the other can check the DC.

@pruivo
Copy link
Member Author

pruivo commented Dec 12, 2013

close for now. I need to change something...

@pruivo pruivo closed this Dec 12, 2013
@pruivo
Copy link
Member Author

pruivo commented Dec 12, 2013

some changes:

  • The EvictCommand is not forward to the primary owner to lock
  • The EvictCommand lock is via ClusterDependendicLogic.lock() method. this allow atomically passivate and activate entries to/from persistence without losing any data
  • Also, it makes theEvictCommand local when compared to previous solution

@pruivo pruivo reopened this Dec 12, 2013
// ensure keys are properly locked for evict commands
command.setFlags(Flag.ZERO_LOCK_ACQUISITION_TIMEOUT, Flag.CACHE_MODE_LOCAL);
try {
lockKey(ctx, command.getKey(), 0, command.hasFlag(Flag.SKIP_LOCKING));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you verified it is safe to no longer lock in a tx for an evict command? I am not aware of why we would require locking other than in passivation, do you know?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see the non tx interceptor did something similar as well.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that code was outdated. Previously, all the nodes acquire the locks for all the operations (non-tx caches) and transactions (tx caches). But then, we introduce the primary owner concept (to reduce the deadlocks) and only one node started to lock.

In that code, it makes no sense to acquire the lock in the backup/non owner because the lock is only acquired in the primary owner.

The synchronization between EvictCommand and operations/transactions is done via CDL.lock()

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree, just wanted to make sure what was going on. Thanks.

@pruivo
Copy link
Member Author

pruivo commented Dec 13, 2013

@wburns updated :)

lock(key, false);
break;
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am thinking if we want this to not support interruption, then just set a boolean variable to symbolize it was interrupted and after this finally acquires the lock, interrupt the thread before returning from this method. Then we don't have to throw a RuntimeException either, which seems suspicious.

* Fixed issues with manual eviction to make it more reliable
* Added tests for manual and size based eviction which happens
concurrently with other operations
@wburns
Copy link
Member

wburns commented Dec 13, 2013

Pulling.

@wburns
Copy link
Member

wburns commented Dec 13, 2013

Integrated, thanks @pruivo !

@wburns wburns closed this Dec 13, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
4 participants