Flush in PersistenceDelegator fails in multi-threaded environment

[yeshvanthni]

A number of Concurrency related issues which are not exactly reproducible in a tescase are seen in load or stress test environment.

The following are a few examples,

java.util.ConcurrentModificationException
at java.util.ArrayDeque$DeqIterator.next(ArrayDeque.java:632)
at java.util.AbstractCollection.toString(AbstractCollection.java:449)
at java.lang.String.valueOf(String.java:2854)
at java.lang.StringBuilder.append(StringBuilder.java:128)
at com.impetus.kundera.persistence.PersistenceDelegator.flush(PersistenceDelegator.java:420)
at com.impetus.kundera.persistence.PersistenceDelegator.persist(PersistenceDelegator.java:175)
at com.impetus.kundera.persistence.EntityManagerImpl.persist(EntityManagerImpl.java:171)
... 10 more

--or--

java.util.NoSuchElementException
at java.util.ArrayDeque.removeFirst(ArrayDeque.java:278)
at java.util.ArrayDeque.pop(ArrayDeque.java:507)
at com.impetus.kundera.persistence.PersistenceDelegator.flush(PersistenceDelegator.java:391)
at com.impetus.kundera.persistence.PersistenceDelegator.persist(PersistenceDelegator.java:151)
at com.impetus.kundera.persistence.EntityManagerImpl.persist(EntityManagerImpl.java:168)
... 14 more

Observations:
-ReentrantReadWriteLock which are acquired for all other operations in PesistenceDelegator is not acquired for flush. It is presumed that the lock would have been acquired in a different method by the same thread prior to reaching flush.

-entitManager.flush() calls flush without acquiring a lock and if a code has persist and explicit flush in a multi-threaded environment this assumption is violated and we are seeing thread errors.

-if by design, flush will not acquire any lock then it should use thread-safe classes like ConcurrentHashMap for eventLogs and ConcurrentLinkedDeque for flushStack.

Let me know if you need further information

[mevivs]

@yeshvanthni

-if by design, flush will not acquire any lock then it should use thread-safe classes like ConcurrentHashMap for eventLogs and ConcurrentLinkedDeque for flushStack.

Thanks for raising this issue. This We are working on it and will be fixed and released in 2.10. Will post an update once done in current trunk.

-Vivek

[mevivs]

A fix has been added for this resolve this issue on trunk.

-Vivek

[pires]

+1

[yeshvanthni]

Could this be made available as a hotfix on an earlier stable release?

[mevivs]

We are planning for a maintenance release(before 2.10). will post an update once we finalize on it's date.

[yeshvanthni]

Thank you for making the fix available. We are continuing to test with snapshot. When can we expect the maintenance release? This is a blocking issue and its being seen within half an hour of starting the application and in 2-3 mins of load testing.

[mevivs]

Possibly by next week. i will post an update on this.

-Vivek

[yeshvanthni]

Is this fix available in 2.9.1 release.?

[mevivs]

Yes. Releasing today with 2.9.1

-Vivek

[mevivs]

Fixed. Releasing with 2.9.1

-Vivek

Note: Group artifactId has been changed. Please refer https://github.com/impetus-opensource/Kundera/blob/trunk/src/README.md#note for the same.

[yeshvanthni]

Thanks