diff --git a/docs/two-phase-commit-transactions.md b/docs/two-phase-commit-transactions.md index 3f1d984cdc..fc228188e4 100644 --- a/docs/two-phase-commit-transactions.md +++ b/docs/two-phase-commit-transactions.md @@ -5,6 +5,21 @@ With Two-phase Commit Transactions, you can execute a transaction that spans mul This document briefly explains how to execute Two-phase Commit Transactions in ScalarDB. +## Overview + +ScalarDB transactions normally execute in a single transaction manager instance. +In that case, you begin a transaction, execute CRUD operations, and commit the transaction in the same transaction manager instance. + +In addition to the normal transactions, ScalarDB also supports two-phase commit style transactions called *Two-phase Commit Transactions*. +Two-phase Commit Transactions execute a transaction that spans multiple transaction manager instances. +The transaction manager instances can be in the same process/applications or in different processes/applications. +For example, if you have transaction manager instances in multiple microservices, you can execute a transaction that spans multiple microservices. + +In Two-phase Commit Transactions, there are two roles, a coordinator and a participant, that collaboratively execute a single transaction. +A coordinator process and participant processes has another transaction manager instance. +The coordinator process first begins a transaction, and the participant processes join the transaction. +And after executing CRUD operations, the coordinator process and the participant processes execute the two-phase commit protocol to commit the transaction. + ## Configuration The configuration for Two-phase Commit Transactions is the same as the one for the normal transaction. @@ -31,11 +46,6 @@ scalar.db.password=cassandra For details about configurations, see [ScalarDB Configurations](configurations.md). -### ScalarDB Server - -You can also execute Two-phase Commit Transactions through the ScalarDB Server. -You don't need a special configuration for Two-phase Commit Transactions, so you can follow [the ScalarDB Server document](scalardb-server.md) to use it. - ## How to execute Two-phase Commit Transactions This section explains how to execute Two-phase Commit Transactions. @@ -46,7 +56,9 @@ The coordinator process first begins a transaction, and the participant processe ### Get a TwoPhaseCommitTransactionManager instance First, you need to get a `TwoPhaseCommitTransactionManager` instance to execute Two-phase Commit Transactions. + You can use `TransactionFactory` to get a `TwoPhaseCommitTransactionManager` instance as follows: + ```java TransactionFactory factory = TransactionFactory.create(""); TwoPhaseCommitTransactionManager transactionManager = factory.getTwoPhaseCommitTransactionManager(); @@ -55,6 +67,7 @@ TwoPhaseCommitTransactionManager transactionManager = factory.getTwoPhaseCommitT ### Begin/Start a transaction (for coordinator) You can begin/start a transaction as follows: + ```java // Begin a transaction TwoPhaseCommitTransaction tx = transactionManager.begin(); @@ -68,6 +81,7 @@ TwoPhaseCommitTransaction tx = transactionManager.start(); The process/application that begins the transaction acts as a coordinator, as mentioned. You can also begin/start a transaction by specifying a transaction ID as follows: + ```java // Begin a transaction with specifying a transaction ID TwoPhaseCommitTransaction tx = transactionManager.begin(""); @@ -88,6 +102,7 @@ tx.getId(); ### Join the transaction (for participants) If you are a participant, you can join the transaction that has been begun by the coordinator as follows: + ```java TwoPhaseCommitTransaction tx = transactionManager.join("") ``` @@ -100,6 +115,7 @@ The CRUD operations of `TwoPhaseCommitTransacton` are the same as the ones of `D So please see also [Java API Guide - CRUD operations](api-guide.md#crud-operations) for the details. This is an example code for CRUD operations in Two-phase Commit Transactions: + ```java TwoPhaseCommitTransaction tx = ... @@ -151,6 +167,7 @@ tx.put(toPut); After finishing CRUD operations, you need to commit the transaction. Like a well-known two-phase commit protocol, there are two phases: prepare and commit phases. You first need to prepare the transaction in all the coordinator/participant processes, and then you need to commit the transaction in all the coordinator/participant processes as follows: + ```java TwoPhaseCommitTransaction tx = ... @@ -172,6 +189,11 @@ try { } ``` +For `preapre()`, if any one of the coordinator or participant processes fails to prepare the transaction, you need to call `rollback()` (or `abort()`) in all the coordinator/participant processes. + +For `commit()`, if any one of the coordinator or participant processes succeeds in committing the transaction, you can consider the transaction as committed. +In other words, in that situation, you can ignore the errors in the other coordinator/participant processes. + If an error happens, you need to call `rollback()` (or `abort()`) in all the coordinator/participant processes. You can call `prepare()`, `commit()`, `rollback()` in the coordinator/participant processes in parallel for better performance. @@ -193,178 +215,88 @@ tx.commit(); ... ``` -Similar to `prepare()`, you can call `validate()` in the coordinator/participant processes in parallel for better performance. +Similar to `prepare()`, if any one of the coordinator or participant processes fails to validate the transaction, you need to call `rollback()` (or `abort()`) in all the coordinator/participant processes. +Also, you can call `validate()` in the coordinator/participant processes in parallel for better performance. Currently, you need to call `validate()` when you use the `Consensus Commit` transaction manager with `EXTRA_READ` serializable strategy in `SERIALIZABLE` isolation level. In other cases, `validate()` does nothing. -### Handle Exceptions +### Execute a transaction with multiple transaction manager instances -Let's look at the following example code to see how to handle exceptions in Two-phase commit transactions. +Let's execute a transaction with multiple transaction manager instances by using the APIs described above: ```java -public class Sample { - public static void main(String[] args) throws IOException, InterruptedException { - TransactionFactory factory = TransactionFactory.create(""); - TwoPhaseCommitTransactionManager transactionManager = - factory.getTwoPhaseCommitTransactionManager(); - - int retryCount = 0; - - while (true) { - if (retryCount++ > 0) { - // Retry the transaction three times maximum in this sample code - if (retryCount >= 3) { - return; - } - // Sleep 100 milliseconds before retrying the transaction in this sample code - TimeUnit.MILLISECONDS.sleep(100); - } - - // Begin a transaction - TwoPhaseCommitTransaction tx; - try { - tx = transactionManager.begin(); - } catch (TransactionNotFoundException e) { - // if the transaction fails to begin due to transient faults. You can retry the transaction - continue; - } catch (TransactionException e) { - // If beginning a transaction failed, it indicates some failure happens during the - // transaction, so you should cancel the transaction or retry the transaction after the - // failure or error is fixed - return; - } - - try { - // Execute CRUD operations in the transaction - Optional result = tx.get(...); - List results = tx.scan(...); - tx.put(...); - tx.delete(...); - - // Prepare the transaction - tx.prepare(); - - // validate the transaction - tx.validate(); - - // Commit the transaction - tx.commit(); - } catch (CrudConflictException - | PreparationConflictException - | ValidationConflictException - | CommitConflictException e) { - // If you catch CrudConflictException or PreparationConflictException or - // ValidationConflictException or CommitConflictException, it indicates a transaction - // conflict occurs during the transaction, so you can retry the transaction from the - // beginning - try { - tx.rollback(); - } catch (RollbackException ex) { - // Rolling back the transaction failed. You can log it here - } - } catch (UnsatisfiedConditionException e) { - // You need to handle UnsatisfiedConditionException only if a mutation operation specifies a condition. - // This exception indicates the condition for the mutation operation is not met, so you can - // retry the transaction once the exception cause is fixed - try { - tx.rollback(); - } catch (RollbackException ex) { - // Rolling back the transaction failed. You can log it here - } - } catch (CrudException | PreparationException | ValidationException | CommitException e) { - // If you catch CrudException or PreparationException or ValidationException or - // CommitException, it indicates some failure happens, so you should cancel the transaction - // or retry the transaction after the failure or error is fixed - try { - tx.rollback(); - } catch (RollbackException ex) { - // Rolling back the transaction failed. You can log it here - } - return; - } catch (UnknownTransactionStatusException e) { - // If you catch `UnknownTransactionStatusException` when committing the transaction, you are - // not sure if the transaction succeeds or not. In such a case, you need to check if the - // transaction is committed successfully or not and retry it if it failed. How to identify a - // transaction status is delegated to users - return; - } - } +TransactionFactory factory1 = + TransactionFactory.create(""); +TwoPhaseCommitTransactionManager transactionManager1 = + factory1.getTwoPhaseCommitTransactionManager(); + +TransactionFactory factory2 = + TransactionFactory.create(""); +TwoPhaseCommitTransactionManager transactionManager2 = + factory2.getTwoPhaseCommitTransactionManager(); + +TwoPhaseCommitTransaction transaction1 = null; +TwoPhaseCommitTransaction transaction2 = null; +try { + // Begin a transaction + transaction1 = transactionManager1.begin(); + + // Join the transaction begun by transactionManager1 with the transaction ID + transaction2 = transactionManager2.join(transaction1.getId()); + + // Execute CRUD operations in the transaction + Optional result = transaction1.get(...); + List results = transaction2.scan(...); + transaction1.put(...); + transaction2.delete(...); + + // Prepare the transaction + transaction1.prepare(); + transaction2.prepare(); + + // Validate the transaction + transaction1.validate(); + transaction2.validate(); + + // Commit the transaction. If any of the transactions succeeds to commit, you can regard the + // transaction as committed + AtomicReference exception = new AtomicReference<>(); + boolean anyMatch = + Stream.of(transaction1, transaction2) + .anyMatch( + t -> { + try { + t.commit(); + return true; + } catch (Exception e) { + exception.set(e); + return false; + } + }); + + // If all the transactions fail to commit, throw the exception and rollback the transaction + if (!anyMatch) { + throw exception.get(); + } +} catch (Exception e) { + // Rollback the transaction + if (transaction1 != null) { + transaction1.rollback(); + } + if (transaction2 != null) { + transaction2.rollback(); } } ``` -The `begin()` API could throw `TransactionException` and `TransactionNotFoundException`. -If you catch `TransactionException`, it indicates some failure (e.g., database failure and network error) happens during the transaction, so you should cancel the transaction or retry the transaction after the failure or error is fixed. -If you catch `TransactionNotFoundException`, it indicates the transaction fails to begin due to transient faults. You can retry the transaction. So you can retry the transaction in this case. - -Although not illustrated in the sample code, the `join()` API could also throw a `TransactionException` and `TransactionNotFoundException`. -And the way to handle them is the same as the `begin()` API. - -The APIs for CRUD operations (`get()`/`scan()`/`put()`/`delete()`/`mutate()`) could throw `CrudException` and `CrudConflictException`. -If you catch `CrudException`, it indicates some failure (e.g., database failure and network error) happens during the transaction, so you should cancel the transaction or retry the transaction after the failure or error is fixed. -If you catch `CrudConflictException`, it indicates a transaction conflict occurs during the transaction so you can retry the transaction from the beginning, preferably with well-adjusted exponential backoff based on your application and environment. -The sample code retries three times maximum and sleeps 100 milliseconds before retrying the transaction. - -The `prepare()` API could throw `PreparationException` and `PreparationConflictException`. -If you catch `PreparationException`, like the `CrudException` case, you should cancel the transaction or retry the transaction after the failure or error is fixed. -If you catch `PreparationConflictException`, like the `CrudConflictException` case, you can retry the transaction from the beginning. - -The `validate()` API could throw `ValidationException` and `ValidationConflictException`. -If you catch `ValidationException`, like the `CrudException` case, you should cancel the transaction or retry the transaction after the failure or error is fixed. -If you catch `ValidationConflictException`, like the `CrudConflictException` case, you can retry the transaction from the beginning. - -The `commit()` API could throw `CommitException`, `CommitConflictException`, and `UnknownTransactionStatusException`. -If you catch `CommitException`, like the `CrudException` case, you should cancel the transaction or retry the transaction after the failure or error is fixed. -If you catch `CommitConflictException`, like the `CrudConflictException` case, you can retry the transaction from the beginning. -If you catch `UnknownTransactionStatusException`, you are not sure if the transaction succeeds or not. -In such a case, you need to check if the transaction is committed successfully or not and retry it if it fails. -How to identify a transaction status is delegated to users. -You may want to create a transaction status table and update it transactionally with other application data so that you can get the status of a transaction from the status table. - -Please note that if you begin a transaction by specifying a transaction ID, you must use a different ID when you retry the transaction. - -Although not illustrated in the sample code, the `resume()` API could also throw a `TransactionNotFoundException`. -This exception indicates that the transaction associated with the specified ID was not found, and it might have been expired. -In such cases, you can retry the transaction from the beginning. - -### Request Routing in Two-phase Commit Transactions +As described above, for `commit()`, if any one of the coordinator or participant processes succeeds in committing the transaction, you can regard the transaction as committed. +Also, you can execute `prepare()`, `validate()`, `commit()` in parallel for better performance. -Services using Two-phase Commit Transactions usually execute a transaction by exchanging multiple requests and responses as follows: - -![](images/two_phase_commit_sequence_diagram.png) +### Resume a transaction -Also, each service typically has multiple servers (or hosts) for scalability and availability and uses server-side (proxy) or client-side load balancing to distribute requests to the servers. -In such a case, since a transaction processing in Two-phase Commit Transactions is stateful, requests in a transaction must be routed to the same servers while different transactions need to be distributed to balance the load. - -![](images/two_phase_commit_load_balancing.png) - -There are several approaches to achieve it depending on the protocol between the services. Here, we introduce some approaches for gRPC and HTTP/1.1. - -#### gPRC - -Please see [this document](https://grpc.io/blog/grpc-load-balancing/) for the details of gRPC Load Balancing. - -When you use a client-side load balancer, you can use the same gRPC connection to send requests in a transaction, which guarantees that the requests go to the same servers. - -When you use a server-side (proxy) load balancer, solutions are different between when using L3/L4 (transport level) and L7 (application level) load balancer. -When using an L3/L4 load balancer, you can use the same gRPC connection to send requests in a transaction, similar to when you use a client-side load balancer. -Requests in the same gRPC connection always go to the same server in L3/L4 load balancing. -When using an L7 load balancer, since requests in the same gRPC connection do not necessarily go to the same server, you need to use cookies or similar for routing requests to correct server. -For example, when you use [Envoy](https://www.envoyproxy.io/), you can use session affinity (sticky session) for gRPC. -Or you can also use [Bidirectional streaming RPC in gRPC](https://grpc.io/docs/what-is-grpc/core-concepts/#bidirectional-streaming-rpc) since the L7 load balancer distributes requests in the same stream to the same server. - -#### HTTP/1.1 - -Typically, you use a server-side (proxy) load balancer with HTTP/1.1. -When using an L3/L4 load balancer, you can use the same HTTP connection to send requests in a transaction, which guarantees the requests go to the same server. -When using an L7 load balancer, since requests in the same HTTP connection do not necessarily go to the same server, you need to use cookies or similar for routing requests to correct server. -You can use session affinity (sticky session) in that case. - -#### Resume a transaction - -Since services using Two-phase Commit Transactions exchange multiple requests/responses, you may need to execute a transaction across multiple endpoints/APIs. -For such cases, you can resume a transaction object (a `TwoPhaseCommitTransaction` instance) that you began or joined as follows: +Given that processes or applications using Two-phase Commit Transactions usually involve multiple request/response exchanges, you might need to execute a transaction across various endpoints or APIs. +For such scenarios, `resume()` is useful, which allows you to resume a transaction object (an instance of `TwoPhaseCommitTransaction`) that you previously began or joined, as follows: ```java // Join (or begin) the transaction @@ -500,6 +432,273 @@ public class ServiceBImpl implements ServiceB { As you can see, by resuming the transaction, you can share the same transaction object across multiple endpoints in `ServiceB`. +### Handle Exceptions + +We already showed [how to execute a transaction with multiple transaction manager instances](#execute-a-transaction-with-multiple-transaction-manager-instances) in the previous section, but we didn't handle exceptions properly there. +In this section, we will show how to handle exceptions in Two-phase commit transactions. + +Two-phase commit transactions are basically executed by multiple processes/applications (a coordinator and participants). +However, in this example code, we use multiple transaction managers (`transactionManager1` and `transactionManager2`) in a single process, for simplicity. + +Let's look at the following example code to see how to handle exceptions in Two-phase commit transactions: + +```java +public class Sample { + public static void main(String[] args) throws Exception { + TransactionFactory factory1 = + TransactionFactory.create(""); + TwoPhaseCommitTransactionManager transactionManager1 = + factory1.getTwoPhaseCommitTransactionManager(); + + TransactionFactory factory2 = + TransactionFactory.create(""); + TwoPhaseCommitTransactionManager transactionManager2 = + factory2.getTwoPhaseCommitTransactionManager(); + + int retryCount = 0; + TransactionException lastException = null; + + while (true) { + if (retryCount++ > 0) { + // Retry the transaction three times maximum in this sample code + if (retryCount >= 3) { + // Throw the last exception if the number of retries exceeds the maximum + throw lastException; + } + + // Sleep 100 milliseconds before retrying the transaction in this sample code + TimeUnit.MILLISECONDS.sleep(100); + } + + TwoPhaseCommitTransaction transaction1 = null; + TwoPhaseCommitTransaction transaction2 = null; + try { + // Begin a transaction + transaction1 = transactionManager1.begin(); + + // Join the transaction begun by transactionManager1 with the transaction ID + transaction2 = transactionManager2.join(transaction1.getId()); + + // Execute CRUD operations in the transaction + Optional result = transaction1.get(...); + List results = transaction2.scan(...); + transaction1.put(...); + transaction2.delete(...); + + // Prepare the transaction + prepare(transaction1, transaction2); + + // Validate the transaction + validate(transaction1, transaction2); + + // Commit the transaction + commit(transaction1, transaction2); + } catch (UnsatisfiedConditionException e) { + // You need to handle `UnsatisfiedConditionException` only if a mutation operation specifies + // a condition. This exception indicates the condition for the mutation operation is not met + + rollback(transaction1, transaction2); + + // You can handle the exception here, according to your application requirements + + return; + } catch (UnknownTransactionStatusException e) { + // If you catch `UnknownTransactionStatusException` when committing the transaction, you are + // not sure if the transaction succeeds or not. In such a case, you need to check if the + // transaction is committed successfully or not and retry it if it failed. How to identify a + // transaction status is delegated to users + return; + } catch (TransactionException e) { + // For other exceptions, you can try retrying the transaction. + + // For `CrudConflictException`, `PreparationConflictException`, + // `ValidationConflictException`, `CommitConflictException` and + // `TransactionNotFoundException`, you can basically retry the transaction. However, for the + // other exceptions, the transaction may still fail if the cause of the exception is + // nontransient. In such a case, you will exhaust the number of retries and throw the last + // exception + + rollback(transaction1, transaction2); + + lastException = e; + } + } + } + + private static void prepare(TwoPhaseCommitTransaction... transactions) + throws TransactionException { + // You can execute `prepare()` in parallel + List exceptions = + Stream.of(transactions) + .parallel() + .map( + t -> { + try { + t.prepare(); + return null; + } catch (TransactionException e) { + return e; + } + }) + .filter(Objects::nonNull) + .collect(Collectors.toList()); + + // If any of the transactions failed to prepare, throw the exception + if (!exceptions.isEmpty()) { + throw exceptions.get(0); + } + } + + private static void validate(TwoPhaseCommitTransaction... transactions) + throws TransactionException { + // You can execute `validate()` in parallel + List exceptions = + Stream.of(transactions) + .parallel() + .map( + t -> { + try { + t.validate(); + return null; + } catch (TransactionException e) { + return e; + } + }) + .filter(Objects::nonNull) + .collect(Collectors.toList()); + + // If any of the transactions failed to validate, throw the exception + if (!exceptions.isEmpty()) { + throw exceptions.get(0); + } + } + + private static void commit(TwoPhaseCommitTransaction... transactions) + throws TransactionException { + // You can execute `commit()` in parallel + List exceptions = + Stream.of(transactions) + .parallel() + .map( + t -> { + try { + t.commit(); + return null; + } catch (TransactionException e) { + return e; + } + }) + .filter(Objects::nonNull) + .collect(Collectors.toList()); + + // If any of the transactions succeeded to commit, you can regard the transaction as committed + if (exceptions.size() < transactions.length) { + if (!exceptions.isEmpty()) { + // You can log the exceptions here if you want + } + + return; // Succeeded to commit + } + + // If all the transactions failed to commit, throw the exception + throw exceptions.get(0); + } + + private static void rollback(TwoPhaseCommitTransaction... transactions) { + Stream.of(transactions) + .parallel() + .filter(Objects::nonNull) + .forEach( + t -> { + try { + t.rollback(); + } catch (RollbackException e) { + // Rolling back the transaction failed. You can log it here + } + }); + } +} +``` + +The `begin()` API could throw `TransactionException` and `TransactionNotFoundException`. +If you catch `TransactionException`, it indicates that the transaction has failed to begin due to transient or nontransient faults. You can try retrying the transaction, but you may not be able to begin the transaction due to nontransient faults. +If you catch `TransactionNotFoundException`, it indicates that the transaction has failed to begin due to transient faults. You can retry the transaction. + +The `join()` API could also throw a `TransactionException` and `TransactionNotFoundException`. +And the way to handle them is the same as the `begin()` API. + +The APIs for CRUD operations (`get()`/`scan()`/`put()`/`delete()`/`mutate()`) could throw `CrudException` and `CrudConflictException`. +If you catch `CrudException`, it indicates that the transaction CRUD operation has failed due to transient or nontransient faults. You can try retrying the transaction from the beginning, but the transaction may still fail if the cause is nontransient. +If you catch `CrudConflictException`, it indicates that the transaction CRUD operation has failed due to transient faults (e.g., a conflict error). You can retry the transaction from the beginning. + +The APIs for mutation operations (`put()`/`delete()`/`mutate()`) could also throw `UnsatisfiedConditionException`. +If you can this exception, it indicates that the condition for the mutation operation is not met. +You can handle this exception according to your application requirements. + +The `prepare()` API could throw `PreparationException` and `PreparationConflictException`. +If you catch `PreparationException`, it indicates that preparing the transaction fails due to transient or nontransient faults. You can try retrying the transaction from the beginning, but the transaction may still fail if the cause is nontransient. +If you catch `PreparationConflictException`, it indicates that preparing the transaction has failed due to transient faults (e.g., a conflict error). You can retry the transaction from the beginning. + +The `validate()` API could throw `ValidationException` and `ValidationConflictException`. +If you catch `ValidationException`, it indicates that validating the transaction fails due to transient or nontransient faults. You can try retrying the transaction from the beginning, but the transaction may still fail if the cause is nontransient. +If you catch `ValidationConflictException`, it indicates that validating the transaction has failed due to transient faults (e.g., a conflict error). You can retry the transaction from the beginning. + +Also, the `commit()` API could throw `CommitException`, `CommitConflictException`, and `UnknownTransactionStatusException`. +If you catch `CommitException`, it indicates that committing the transaction fails due to transient or nontransient faults. You can try retrying the transaction from the beginning, but the transaction may still fail if the cause is nontransient. +If you catch `CommitConflictException`, it indicates that committing the transaction has failed due to transient faults (e.g., a conflict error). You can retry the transaction from the beginning. +If you catch `UnknownTransactionStatusException`, it indicates that the status of the transaction, whether it has succeeded or not, is unknown. +In such a case, you need to check if the transaction is committed successfully or not and retry it if it fails. +How to identify a transaction status is delegated to users. +You may want to create a transaction status table and update it transactionally with other application data so that you can get the status of a transaction from the status table. + +Although not illustrated in the sample code, the `resume()` API could also throw a `TransactionNotFoundException`. +This exception indicates that the transaction associated with the specified ID was not found, and it might have been expired. +In such cases, you can retry the transaction from the beginning since the cause of this exception is basically transient. + +In the sample code, for `UnknownTransactionStatusException`, the transaction doesn't retry because the cause of the exception is nontransient. +Also, for `UnsatisfiedConditionException`, the transaction doesn't retry because how to handle this exception depends on your application requirements. +For other exceptions, the transaction tries retrying because the cause of the exception is transient or nontransient. +If the cause of the exception is transient, the transaction may succeed if you retry it. +However, if the cause of the exception is nontransient, the transaction may still fail even if you retry it. +In such a case, you will exhaust the number of retries. + +Please note that if you begin a transaction by specifying a transaction ID, you must use a different ID when you retry the transaction. +And, in the sample code, the transaction retries three times maximum and sleeps for 100 milliseconds before it retries. +But you can choose a retry policy, such as exponential backoff, according to your application requirements. + +## Request Routing in Two-phase Commit Transactions + +Services using Two-phase Commit Transactions usually execute a transaction by exchanging multiple requests and responses as follows: + +![](images/two_phase_commit_sequence_diagram.png) + +Also, each service typically has multiple servers (or hosts) for scalability and availability and uses server-side (proxy) or client-side load balancing to distribute requests to the servers. +In such a case, since a transaction processing in Two-phase Commit Transactions is stateful, requests in a transaction must be routed to the same servers while different transactions need to be distributed to balance the load. + +![](images/two_phase_commit_load_balancing.png) + +There are several approaches to achieve it depending on the protocol between the services. Here, we introduce some approaches for gRPC and HTTP/1.1. + +### gPRC + +Please see [this document](https://grpc.io/blog/grpc-load-balancing/) for the details of gRPC Load Balancing. + +When you use a client-side load balancer, you can use the same gRPC connection to send requests in a transaction, which guarantees that the requests go to the same servers. + +When you use a server-side (proxy) load balancer, solutions are different between when using L3/L4 (transport level) and L7 (application level) load balancer. +When using an L3/L4 load balancer, you can use the same gRPC connection to send requests in a transaction, similar to when you use a client-side load balancer. +Requests in the same gRPC connection always go to the same server in L3/L4 load balancing. +When using an L7 load balancer, since requests in the same gRPC connection do not necessarily go to the same server, you need to use cookies or similar for routing requests to correct server. +For example, when you use [Envoy](https://www.envoyproxy.io/), you can use session affinity (sticky session) for gRPC. +Or you can also use [Bidirectional streaming RPC in gRPC](https://grpc.io/docs/what-is-grpc/core-concepts/#bidirectional-streaming-rpc) since the L7 load balancer distributes requests in the same stream to the same server. + +### HTTP/1.1 + +Typically, you use a server-side (proxy) load balancer with HTTP/1.1. +When using an L3/L4 load balancer, you can use the same HTTP connection to send requests in a transaction, which guarantees the requests go to the same server. +When using an L7 load balancer, since requests in the same HTTP connection do not necessarily go to the same server, you need to use cookies or similar for routing requests to correct server. +You can use session affinity (sticky session) in that case. + ## Further reading One of the use cases for Two-phase Commit Transactions is Microservice Transaction.