Alternator: support "transactions" feature

[nyh]

"Transactions" are a new DynamoDB feature (announced in November 2018) which allows atomic (all-or-nothing) updates to one or more tables in the same region. This feature is unrelated to the older single-item conditional updates feature - which in Scylla are called lightweight transactions, and which we already support but without the correct isolation guarantees (see issue #5054).

See https://docs.google.com/document/d/1i4yjF5OSAazAY_-T8CBce9-2ykW4twx_E_Nt2zDoOVs/edit# (section "Transactions") for some implementation ideas.

The new API requests for this feature: TransactWriteItems and TransactGetItems.

[nyh]

In https://docs.google.com/document/d/1i4yjF5OSAazAY_-T8CBce9-2ykW4twx_E_Nt2zDoOVs/edit#heading=h.qmq6kez54pdq I mentioned that:

Before this feature [transactions] was added, users were encouraged to use Amazon’s transaction client library which implements transactions on top of other DynamoDB features. We can encourage them to do the same. Perhaps eventually we can take ideas from this library on how to implement transactions on the server side.

This may be a good way to have full support for this feature - albeit somewhat ugly and slow - quickly, without huge investments in Scylla infrastructures to support transactions in CQL. Amazon have good design document https://github.com/awslabs/dynamodb-transactions/blob/master/DESIGN.md on how they implemented full-fledged multi-item cross-table transactions over pre-existing DynamoDB features, and I'm optimistic we can do the same while emulating their new transaction API. I'm optimistic that this can be done in 1-2 weeks instead of the original 1-2 month estimate (which itself was optimistic) for a from-scratch implementation of this feature.

[nyh]

I summarized the option of providing transactions over existing features in a new design document https://docs.google.com/document/d/1d0RmuWPkGmoKPXwJ2mY6HLhf40otuMvfOKNUYcfn2Cg/edit#. There are good news and bad news. The good news is that this I think this technique can work. The bad news is that the performance will be bad (e.g., a transactional write will be 7-9 times more expensive than a regular LWT write, while the ratio is 2 in DynamoDB), and some edge cases involving races of transactional and non-transactional operations, or interaction of transactions and GSI or streams, will be tricker to solve.

[nyh]

Our LWT implementation already supports conditional batches: "The entire conditional batch has an isolated view of the database and is executed using all-or-nothing principle.".

We could use this existing capability to implement the DynamoDB Transactions feature with one important limitation - that all operations in a transaction must be on the same partition. Although this limited feature is not as powerful as the full DynamoDB feature (that can cross partitions and even tables), it will probably be good enough for some users - and we should hopefully be able to provide this subset quickly and most importantly - efficiently (it will have the same performance as the current CAS-based writes).

[nyh]

Amazon's FAST '19 presentation by Doug Terry on how DynamoDB transactions were implemented: https://www.usenix.org/conference/fast19/presentation/terry
A nice summary of their approach and why it is simpler than other possible approaches: https://www.mydistributed.systems/2021/12/dynamodb.html

Some additional interesting links:

• DynamoDB Transactions examples and use cases, by Alex DeBrie (a consultant who also wrote the DynamoDB book): https://www.alexdebrie.com/posts/dynamodb-transactions/
• DynamoDB Transactions performance tests - comparing latencies of transactions to regular batch write (note that the cost is already known - double, so the remaining performance issue is the latency). https://www.alexdebrie.com/posts/dynamodb-transactions-performance/

[jexh]

Is there a core scylladb issue describing the requirements to support this issue? I can't find anything that requests cross-table transaction support in the issue list - perhaps I just can't think of the correct search phrase?

[nyh]

Is there a core scylladb issue describing the requirements to support this issue?

I don't think there is such an issue open. I agree that it would also be nice to have multi-item transactions in CQL and that whatever core feature we'll end up implementing may also be used in CQL to implement a similar transaction feature. The reason why I opened this issue specifically for Alternator (ScyllaDB's DynamoDB-compatible API) is that it is a requirement for full DynamoDB compatibility, and that feature has known behavior (defined by Amazon). For the CQL feature it would be a completely new feature and we'll need to design it and determine how exactly it should behave. Or maybe it should be compatible with the planned Cassandra feature - https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-15%3A+General+Purpose+Transactions (a.k.a. "Accord").

UPDATE: CEP-15: General Purpose Transactions was officially accepted in Cassandra, there is an issue - https://issues.apache.org/jira/browse/CASSANDRA-17092, and will probably miss Cassandra 5.0 and be in Cassandra 5.1. So to maintain Cassandra compatibility, the CQL feature was already defined for us.

[jexh]

Thanks for the background. From what you say, it seems to me that it would now be useful to open a separate issue for multi-item transaction support in CQL. Such an issue would be a good place to capture requirements like supporting this Alternator extension but also provide a place for discussing the CQL specific design aspects. I think that such a feature is very desirable in its own right and the probability of starting to really tackle issues such as this one will be much higher if we really recognize the benefits it could bring to the whole product.

…

On Sun, 29 Jan 2023 at 09:26, nyh ***@***.***> wrote: Is there a core scylladb issue describing the requirements to support this issue? I don't think there is such an issue open. I agree that it would also be nice to have multi-item transactions in CQL and that whatever core feature we'll end up implementing may also be used in CQL to implement a similar transaction feature. The reason why I opened this issue specifically for Alternator (ScyllaDB's DynamoDB-compatible API) is that it is a requirement for full DynamoDB compatibility, and that feature has known behavior (defined by Amazon). For the CQL feature it would be a completely new feature and we'll need to design it and determine how exactly it should behave. Or maybe it should be compatible with the planned Cassandra feature - https://cwiki.apache.org/confluence/display/CASSANDRA/CEP-15%3A+General+Purpose+Transactions — Reply to this email directly, view it on GitHub <#5064 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AADWOAUPPSHHGG2XPNC6PJTWUYZT3ANCNFSM4IYIEL4Q> . You are receiving this because you commented.Message ID: ***@***.***>

[nyh]

Originally, DynamoDB transactions were limited to contain 25 read or write requests. Starting in September 2022, this limit was increased to 100 operations in a single transaction.