Provide a "best efforts" 1PC transaction manager out of the box [SPR-3844]

[spring-issuemaster]

Juergen Hoeller opened SPR-3844 and commented

There have been detailed discussions about a "best efforts" transaction strategy that allows any number of local resource transactions to be part of a single Spring-managed transaction. Spring 2.0 already supports synchronized local JMS transactions in that style; this issue is effectively about extending that approach to all of Spring's supported resources.

Important 'side' features are commit ordering and customizable transaction logging (see #7996). This JIRA issue (together with #7996) serves for tracking requirements, suggestions and contributions, as well as the status of the "best efforts" transaction manager implementation in Spring.

---

Issue Links:

• DATACMNS-310 Integrate best-effort TransactionManager from Spring Data Neo4j
• #18308 Move ChainedTransactionManager from spring-data commons to spring-tx
• DATAGRAPH-256 Move ChainedTransactionManager from Spring Data Neo4j to Spring Framework

68 votes, 60 watchers

[spring-issuemaster]

Juergen Hoeller commented

A bit of background and motivation for this planned feature...

Multi-resource transactions may:

1. either use JTA transactions, accessing XA-enabled DataSources
2. or use a combination of 1..x interleaved native transactions.

The first variant can quite easily be done using Spring's JtaTransactionManager with proper JTA/XA setup in the backend. Plain @Transactional annotations will work fine here, since they operate against the single central JtaTransactionManager. This will work in a J2EE application server as well as in Tomcat or standalone setups (with an embedded JTA provider). Depending on the JTA provider, this may also work with interleaved local transactions for non-XA resources. However, this is not too common and usually not straightforward to setup. That aside, there may be the explicit desire to avoid the XA protocol, preferring the more efficient processing of native transactions over 2PC atomicity and recovery guarantees.

The second option is essentially what this issue is about. This is what some JTA providers support for non-XA resources (as outlined above). We intend to provide this in a different form: through a best-efforts transaction manager (non-JTA) that is able to 'enlist' all native resources that have been touched during an application's transaction, performing a sequence of native commit calls at the end. This allows for using a single shared transaction manager, with no need to identify a specific transaction manager in the @Transactional annotation, while still supporting a variety of native resources - as actually touched within the transactions at runtime. This can be refined with customizable transaction logging etc.

[spring-issuemaster]

Chuck Canning commented

I have implemented a temporary solution based on option 2 that is working. It is a little funky because it couldn't completely integrate due to the fact that you cast to implementations in the Abstract classes instead of purely using interfaces. I can provide this to anyone who is interested and if you want, you can use it as a prototype for the development of the real solution (which I hope comes soon).

[spring-issuemaster]

Ronald van Rijn commented

I have given this issue/feature some thought after a small discussion I had with Juergen at SpringOne.

The problem with enlisting all native resources that have been touched is that you do not know the order in which the native transactions have to be committed. Therefore there either needs to be a way to specify the commit order, or there needs to be a well defined default order (preferably both).

A way to specify the commit order could be having primary and secondary resources on the transaction manager, like this:

<bean class="...BestEffortTransactionManager">
<property name="primaryResource" ref="datasource" />
<property name="secondaryResource" ref="connectionFactory" />
</bean>

or by specifying some kind of priority list:

<bean class="...BestEffortTransactionManager">
<list>
<ref bean="datasource" />
<ref bean="connectionFactory" />
</list>
</bean>

Depending on the resources that are registered with the transaction manager and the order in which they are specified the transaction manager will know how to complete the transaction.

Maybe a sensible default for the order in which the native transactions will be committed can be the reverse order in which the native transactions were started.

For example: if I would receive a message through a jms listener, do some database work in between, and send a message to confirm that I finished my task, the commit order would be:

1. the database (the native transaction that started last)
2. the connection factory (the native transaction that started first)

For my example this would be a very sensable default, I'm not sure about using multiple databases in one transaction though. But as long as the default behaviour is documented and there is a way to override this default behaviour through some means of configuration I think this is a very usable solution and I would definitely use it when it would become available.

[spring-issuemaster]

Chuck Canning commented

There also needs to be a way to specify how the transactions commit after the first and subsequent failures. We have a case where 1/n commits is fine (all can fail besides any one) and a case where 1 + 0/n where if the first one fails, the rest rollback and any subsequent failures can be ignored. This could probbaly be controlled by the best effort interface.

We would also want to specify that certain resources always commit first (ie. we do not allow users to make a JMS call inside a DB transaction, but we may want both transactions to be managed together - db fails JMS).

You may also want to specify that certain resources can't be used together (ie. JMS and DB).

Also, using pure interfaces may help (spring 2.5.x uses abstract impls internally which prevented us from properly implementing our own solution at certain levels).

[spring-issuemaster]

Ronald van Rijn commented

There also needs to be a way to specify how the transactions commit after the first and subsequent failures. We have a case where 1/n commits is fine (all can fail besides any one) and a case where 1 + 0/n where if the first one fails, the rest rollback and any subsequent failures can be ignored. This could probbaly be controlled by the best effort interface.

I think the best effort transaction manager should mimic the behaviour of the XA protocol. This means that if the commit on one resource fails all resources after that need to be rolled back.

If, in your use case, there are resources that always need to be committed (even when a commit on an other resource failed) you have to access these resources in a separate transaction.

We would also want to specify that certain resources always commit first (ie. we do not allow users to make a JMS call inside a DB transaction, but we may want both transactions to be managed together - db fails JMS).

This is what I meant with the commit order in my previous comment. If you want the transactions on a certain resource to be committed first you put this resource higher in the order of managed resources.

[spring-issuemaster]

Donnchadh O Donnabhain commented

With 31 votes and 34 watchers maybe we can put together a pull request for this. Are many of the watchers still interested in this?
Dave Syer's simple implementation of a few years ago is described at http://www.javaworld.com/javaworld/jw-01-2009/jw-01-spring-transactions.html?page=5 with sample code at http://www.javaworld.com/javaworld/jw-01-2009/springxa-src.zip

[spring-issuemaster]

Donnchadh O Donnabhain commented

See also SPR-9045 .

[spring-issuemaster]

Giovanni Botta commented

Hey guys, this is exactly my use case. I'd like to have a best effort 1PC transaction manager available in Spring.

[spring-issuemaster]

Giovanni Botta commented

Could this be used as a patch (or a baseline for a patch): https://jira.springsource.org/browse/SPR-6237 ?

[spring-issuemaster]

Giovanni Botta commented

I'm actually trying to use the code in #10905 but I think it can (and should) be improved before making it a patch.

[spring-issuemaster]

Giovanni Botta commented

Update: the code doesn't seem to be working with version 3+, same for the one written by Dave Syer.

[spring-issuemaster]

Donnchadh O Donnabhain commented

How about ChainedTransactionManager from Spring Data Neo4j mentioned in DATAGRAPH-256?
https://github.com/SpringSource/spring-data-neo4j/blob/master/spring-data-neo4j-tx/src/main/java/org/springframework/data/neo4j/transaction/ChainedTransactionManager.java

[spring-issuemaster]

Giovanni Botta commented

I'll check that out. Thanks!

[spring-issuemaster]

Giovanni Botta commented

This seems to be working fine, however a major bug was reported: https://jira.springsource.org/browse/DATAGRAPH-191

[spring-issuemaster]

Oliver Drotbohm commented

I've just fixed the bug in DATAGRAPH-191 and moved the transaction manager implementation into Spring Data Commons as it is not strictly tied to graph database scenarios. We're going to deprecate the class in Neo4j for the next major release and remove it in the next but one release.

[spring-issuemaster]

Giovanni Botta commented

Thanks Oliver. Which Spring release will this be included with?

[spring-issuemaster]

Oliver Drotbohm commented

It will not be in Spring in the first place (although it might be considered for the 4.0 timeframe). We'll see it in Spring Data Commons 1.6 M1, scheduled for end of May. The relevant ticket is here: DATACMNS-310.

The bugfix will already released with Spring Data Graph as part of the Spring Data Arora SR1 release due this week.

[spring-issuemaster]

Giovanni Botta commented

I didn't realize Spring Data Commons was a separate projects. Thanks