Support for Portable Transactions

[jimmarino]

Overview

This issue outlines the requirements for, and the implementation of, transaction support in the EDC.

The main requirements for transactions are:

1. It must be possible to support transactional operations (create, update, delete, enqueue, dequeue) on data in the runtime.

2. Transactional operations may span multiple services contributed to the runtime by different extensions.

3. Transactional operations may be composed differently based on use cases and code execution paths. For example, consider the following situations:

Service A ---> Service 1 ([work])

In this case, Service A invokes Service 1 in insolation. Service 1 demarcates the work it performs within a transaction, denoted with [..].

Service A [----> Service 1 (work), Service 2 (work)]

In this case, all services demarcate their work within transactions. Since Service A defines a transactional boundary that spans invocations to Service 1 and Service 2, their transactions join the parent Service A transaction

4. Transaction support most offer a portable programming model, whether the underlying implementation is based on a JTA transaction manager, a local resource such as a JDBC connection pool, or a host environment such as Spring.

Core Services

TransactionContext

The transaction implementation will be centered on a TransactionContext registered with the runtime service context:

public interface TransactionContext {

    /**
     * Executes the code within a transaction.
     */
    void execute(TransactionBlock block);

    /**
     * Defines a block of transactional code.
     */
    @FunctionalInterface
    interface TransactionBlock {

        /**
         * Executes the transaction block.
         */
        void execute();

    }
}

The TransactionContext provides a portable programming model by abstracting the specifics of transaction management. It also gives the added benefit of handling joining, completion, and rollback operations, which are repetitive and error-prone tasks. The following demonstrates how code interacts with the TransactionContext, including the semantics of nested transaction demarcation:

        transactionContext.execute(() -> {
            // top level work...
            transactionContext.execute(() -> {
                // nested work joins the parent transaction...
            });
            // transaction work completes
        });
    }

Underlying transaction subsystems provide an implementation of TransactionContext. There will initially be an implementation that supports local transactions. A JTA-based version can also be implemented in a subsequent PR.

Note that begin(), commit(), rollback() and setRollbackOnly() methods could be added to TransactionContext in a future PR. This may be useful for bulk loading scenarios where batches must be transactionally committed. In general, however, transaction blocks should be preferred as it alleviates user-code from needing to manage transaction operations.

DataSourceRegistry

A DataSourceRegistry will be introduced that provides a portable way to resolve DataSources in the EDC, regardless of whether local or global transactions are enabled:

/**
 * A registry of configured data sources.
 */
public interface DataSourceRegistry {

    /**
     * The default data source.
     */
    String DEFAULT_DATASOURCE = "default";

    /**
     * Registers a datasource.
     */
    void register(String name, DataSource dataSource);

    /**
     * Returns the datasource registered for the name or null if not found.
     */
    @Nullable
    DataSource resolve(String name);

}

DataSource extensions will initialize one or more DataSource instances and register them. Extensions can then resolve a required instance using the registry. As a convention, extensions should offer a data source configuration option, and if not specified, use the default data source.

Local Transactions

A local transaction implementation will be provided. When a single data source (or more generally, a single transactional resource) is used for all transactional execution, ACID semantics will be guaranteed across services. If multiple transactional resources are used, there are no ACID guarantees; all enlisted resources will be committed sequentially. This may result in partial updates.

To integrate a transactional resource with the local transaction implementation, an extension must provide a service that implements LocalTransactionResource:

/**
 * A resource that can be enlisted in a local transaction.
 */
public interface LocalTransactionResource {

    /**
     * Signals a transaction has started.
     */
    void start();

    /**
     * Signals a transaction has committed.
     */
    void commit();

    /**
     * Signals a transaction has been rolled back.
     */
    void rollback();

}

A local transaction resource is registered using the LocalTransactionContextManager:

/**
 * Registers transactional resources so they can be coordinated.
 */
public interface LocalTransactionContextManager {

    /**
     * Registers the resource.
     */
    void registerResource(LocalTransactionResource resource);

}

A default LocalTransactionResource implementation will be provided for JDBC DataSources. This will make it trivial for a data source extension (such as a connection pool) to register itself and be enlisted in a local transaction. It will only need to instantiate a DataSource and register it with the DataSourceRegistry:

 DataSource dataSource = .... // e.g., instantiate a pool
 
// automatically registers the data source with the LocalTransactionContextManager 
 registry.register(DataSourceRegistry.DEFAULT_DATASOURCE, dataSource)); 

[jimmarino]

The PR is #540.

[DominikPinsel]

DataSource extensions will initialize one or more DataSource instances and register them.

I assume it will be a problem to have two data sources of the same type, if there exists an extension per data source. For example two SQL data sources (e.g. PostgreSQL). Maybe you can further explain how this can be addressed.

_{Dominik Pinsel dominik.pinsel@daimler.com, Daimler TSS GmbH, legal info/Impressum}

[jimmarino]

DataSource extensions will initialize one or more DataSource instances and register them.

I assume it will be a problem to have two data sources of the same type, if there exists an extension per data source. For example two SQL data sources (e.g. PostgreSQL). Maybe you can further explain how this can be addressed.

Dominik Pinsel dominik.pinsel@daimler.com, Daimler TSS GmbH, legal info/Impressum

It shouldn't be a problem - that case is specifically handled in the proposal. Just register the two DS instances under different names. The DataSource extension should be able to instantiate multiple DSes and register then.

[DominikPinsel]

That's true. I also think that there will be no issue to this point. Maybe I need to explain it better. I'd assume it will be difficult to create and configure extensions for each data source.

If there is an extension for an PostgreSQL DataSource, each PostgreSQL data source will need to be configured individually. Using the current setting concept, with key-value based settings, I don't see how it will be possible to configure the extension accordingly.
For example to configure two different kinds of data source, this extension would need to know separate values for keys like

• edc.datasource.postgreql.url
• edc.datasource.postgresql.user
• edc.datasource.postgresql.password

_{Dominik Pinsel dominik.pinsel@daimler.com, Daimler TSS GmbH, legal info/Impressum}

[jimmarino]

This can be done by creating a naming convention and adding one additional method to ServiceExtensionContext. We need the ability to set extensible properties that the DS implementation is not aware of as well since JDBC drivers have custom configuration. Both requirements can be satisfied by this approach.

Let me take a stab at this in conjunction with the current PR.