WriteConflict when using Mongo transactions

[jkloecknerneuland]

When using Javers with MongoDB and transaction support enabled, writing multiple commits at the same yields a WriteConflict error. An expected behavior would probably be to retry the commit or transaction.

Steps To Reproduce
I have a runnable test case which isolates the bug and allows Javers Core Team to easily reproduce it. I have pushed this test case to my fork of this repository:
https://github.com/jkloecknerneuland/javers/blob/master/javers-spring-boot-starter-mongo/src/test/groovy/org/javers/spring/boot/mongo/JaversMultipleTransactionsMongoTest.groovy

Javers' Version
6.6.5

Additional context
This is probably because the counter in the jv_head_id collection cannot be updated, because its blocked by another transaction.

[Zakhar2093]

I got the same bug. It was really difficult to find the cause. I have just the usual Controller which only inserts one document into the collection. When the backend performs several simultaneous requests I got this error. The solution was to delete javers-spring-boot-starter-mongo from pom.xml

[jkloecknerneuland]

@Zakhar2093 Thanks for the suggestion, but unfortunately removing the javers-spring-boot-starter-mongo dependency is not an option for me, as I need the provided functionality of it.

[jkloecknerneuland]

@bartoszwalacik
I created a pull request (#1209) with a test case and a possible fix for this issue, could you review it?

[jkloecknerneuland]

Sadly, this problem prevents me from using Javers, but I would like to contribute to the project to fix this.
Is there anything I can do?

[kulasekp]

Hi,
@bartoszwalacik @jkloecknerneuland
We have faced exactly the same issue: Mongo transaction ends with WriteConflict because it fails to update jv_head_id:

	at org.javers.repository.mongo.MongoRepository.transactionalUpdate(MongoRepository.java:413)
	at org.javers.repository.mongo.MongoRepository.persistHeadId(MongoRepository.java:256)
	at org.javers.repository.mongo.MongoRepository.persist(MongoRepository.java:99)
	at org.javers.spring.mongodb.TransactionalMongoRepository.lambda$persist$0(TransactionalMongoRepository.java:76)
	at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:140)
	at org.javers.spring.mongodb.TransactionalMongoRepository.persist(TransactionalMongoRepository.java:74)
	at org.javers.repository.api.JaversExtendedRepository.persist(JaversExtendedRepository.java:154)

@jkloecknerneuland
We were considering similar solution, just on our service side, with Spring Retry, but we quickly realized it would be very problematic for various reasons: depending on the configuration whole transactional code (so not the only one line you would actually expect) will be executed once again, the changes might be saved in wrong order or rejected completely (depending on if you are using pessimistic/optimistic/no lock) or just retried few times without final success and just dropped.

So, my point is that adding retry for persist, which at a glance looks quite ok, might not be the best solution here.

@bartoszwalacik
I would like to also have it fixed and provide some proposals here but first would be great to understand what is the purpose of "jv_head_id".

For me it looks like some kind custom made database sequence, similar to TABLE identifier generator in JPA.
And this reminds me all the tricks you need to do to have it working with concurrency in mind: in general operation to generate id should be as atomic as possible, synchronized, with lock on the document (sql equivalent: select for update). Maybe even executed in dedicated transaction, like REQUIRES_NEW in Spring?

The other approach, I am also thinking is if it would be possible to move to append only mode, so instead of updating one document, just create new ones (if I am not mistaken -> similarly to SQL version of JaVers)??

PS
Aaand finally I just noticed CommitIdFactory and its default configuration set to SYNCHRONIZED_SEQUENCE with comment "Should not be used in distributed applications." -> CommitIdGenerator
Maybe switching to RANDOM (should be possible via configuration as per #688) will also help here in some way?

[kulasekp]

As we might have suspected : RANDOM won't help at all.
But still question: as RANDOM generates next jv_head_id without taking previous value of jv_head_id into account (no db call to get it)... is it even necessary to save jv_head_id to the db in this case? If not then at least for RANDOM "WriteConflict" should not occur anymore.

As for now: temporarily I finished with disabling transactions for Javers Mongo and not it works fine. Of course, it should not be considered as target solution but... would you be fine with adding additional configuration parameter allowing to disable transaction management on user's will? Currently Javers automatically enables transaction management once transaction manager is found but it might be some users would like to retain non-transactional approach for some reason (as in my case).
It can be done by configuring Javers manually (as I did) but it would be far more convenient to just do it via property.
I can propose PR, but first would like to know your opinion on the matter.

[bartoszwalacik]

Hi guys, I will take a look at this case, sorry for the delay. Skipping update of jv_head_id for RANDOM seems like a quick win

[bartoszwalacik]

fix 8709929

[bartoszwalacik]

this test shows the working config
https://github.com/javers/javers/blob/master/javers-spring-boot-starter-mongo/src/test/groovy/org/javers/spring/boot/mongo/JaversConcurrentTransactionsMongoTest.groovy

the head_id collection is now skipped when RANDOM commitId generator is selected

javers:
    commitIdGenerator: random

[bartoszwalacik]

fix released in 7.3.0