Commander explanation

commander/README.md

@@ -1,25 +1,162 @@
 ---
 title: Commander
-category: Concurrency
+category: Behavioral
 language: en
 tag:
- - Cloud distributed
+    - Cloud distributed
+    - Microservices
+    - Transactions
 ---
 
+## Also known as
+
+* Distributed Transaction Commander
+* Transaction Coordinator
+
 ## Intent
 
-> Used to handle all problems that can be encountered when doing distributed transactions.
+The intent of the Commander pattern in the context of distributed transactions is to manage and coordinate complex
+transactions across multiple distributed components or services, ensuring consistency and integrity of the overall
+transaction. It encapsulates transaction commands and coordination logic, facilitating the implementation of distributed
+transaction protocols like two-phase commit or Saga.
+
+## Explanation
+
+Real-world example
+
+> Imagine organizing a large international music festival where various bands from around the world are scheduled to
+> perform. Each band's arrival, soundcheck, and performance are like individual transactions in a distributed system. The
+> festival organizer acts as the "Commander," coordinating these transactions to ensure that if a band's flight is
+> delayed (akin to a transaction failure), there's a backup plan, such as rescheduling or swapping time slots with another
+> band (compensating actions), to keep the overall schedule intact. This setup mirrors the Commander pattern in
+> distributed transactions, where various components must be coordinated to achieve a successful outcome despite
+> individual failures.
+
+In plain words
+
+> The Commander pattern turns a request into a stand-alone object, allowing for the parameterization of commands,
+> queueing of actions, and the implementation of undo operations.
+
+**Programmatic Example**
+
+Managing transactions across different services in a distributed system, such as an e-commerce platform with separate
+Payment and Shipping microservices, requires careful coordination to avoid issues. When a user places an order but one
+service (e.g., Payment) is unavailable while the other (e.g., Shipping) is ready, we need a robust solution to handle
+this discrepancy.
+
+A strategy to address this involves using a Commander component that orchestrates the process. Initially, the order is
+processed by the available service (Shipping in this case). The commander then attempts to synchronize the order with
+the currently unavailable service (Payment) by storing the order details in a database or queueing it for future
+processing. This queueing system must also account for possible failures in adding requests to the queue.
+
+The commander repeatedly tries to process the queued orders to ensure both services eventually reflect the same
+transaction data. This process involves ensuring idempotence, meaning that even if the same order synchronization
+request is made multiple times, it will only be executed once, preventing duplicate transactions. The goal is to achieve
+eventual consistency across services, where all systems are synchronized over time despite initial failures or delays.
+
+In the provided code, the Commander pattern is used to handle distributed transactions across multiple services (
+PaymentService, ShippingService, MessagingService, EmployeeHandle). Each service has its own database and can throw
+exceptions to simulate failures.
+
+The Commander class is the central part of this pattern. It takes instances of all services and their databases, along
+with some configuration parameters. The placeOrder method in the Commander class is used to place an order, which
+involves interacting with all the services.
+
+```java
+public class Commander {
+    // ... constructor and other methods ...
+
+    public void placeOrder(Order order) {
+        // ... implementation ...
+    }
+}
+```
+
+The User and Order classes represent a user and an order respectively. An order is placed by a user.
+
+```java
+public class User {
+    // ... constructor and other methods ...
+}
+
+public class Order {
+    // ... constructor and other methods ...
+}
+```
+
+Each service (e.g., PaymentService, ShippingService, MessagingService, EmployeeHandle) has its own database and can
+throw exceptions to simulate failures. For example, the PaymentService might throw a DatabaseUnavailableException if its
+database is unavailable.
+
+```java
+public class PaymentService {
+    // ... constructor and other methods ...
+}
+```
+
+The DatabaseUnavailableException, ItemUnavailableException, and ShippingNotPossibleException classes represent different
+types of exceptions that can occur.
+
+```java
+public class DatabaseUnavailableException extends Exception {
+    // ... constructor and other methods ...
+}
+
+public class ItemUnavailableException extends Exception {
+    // ... constructor and other methods ...
+}
+
+public class ShippingNotPossibleException extends Exception {
+    // ... constructor and other methods ...
+}
+```
+
+In the main method of each class (AppQueueFailCases, AppShippingFailCases), different scenarios are simulated by
+creating instances of the Commander class with different configurations and calling the placeOrder method.
 
 ## Class diagram
+
 ![alt text](./etc/commander.urm.png "Commander class diagram")
 
 ## Applicability
-This pattern can be used when we need to make commits into 2 (or more) databases to complete transaction, which cannot be done atomically and can thereby create problems.
 
-## Explanation
-Handling distributed transactions can be tricky, but if we choose to not handle it carefully, there could be unwanted consequences. Say, we have an e-commerce website which has a Payment microservice and a Shipping microservice. If the shipping is available currently but payment service is not up, or vice versa, how would we deal with it after having already received the order from the user?
-We need a mechanism in place which can handle these kinds of situations. We have to direct the order to either one of the services (in this example, shipping) and then add the order into the database of the other service (in this example, payment), since two databases cannot be updated atomically. If currently unable to do it, there should be a queue where this request can be queued, and there has to be a mechanism which allows for a failure in the queueing as well. All this needs to be done by constant retries while ensuring idempotence (even if the request is made several times, the change should only be applied once) by a commander class, to reach a state of eventual consistency.
+Use the Commander pattern for distributed transactions when:
+
+* You need to ensure data consistency across distributed services in the event of partial system failures.
+* Transactions span multiple microservices or distributed components requiring coordinated commit or rollback.
+* You are implementing long-lived transactions requiring compensating actions for rollback.
+
+## Known Uses
+
+* Two-Phase Commit (2PC) Protocols: Coordinating commit or rollback across distributed databases or services.
+* Saga Pattern Implementations: Managing long-lived business processes that span multiple microservices, with each step
+  having a compensating action for rollback.
+* Distributed Transactions in Microservices Architecture: Coordinating complex operations across microservices while
+  maintaining data integrity and consistency.
+
+## Consequences
+
+Benefits:
+
+* Provides a clear mechanism for managing complex distributed transactions, enhancing system reliability.
+* Enables the implementation of compensating transactions, which are crucial for maintaining consistency in long-lived
+  transactions.
+* Facilitates the integration of heterogeneous systems within a transactional context.
+
+Trade-offs:
+
+* Increases complexity, especially in failure scenarios, due to the need for coordinated rollback mechanisms.
+* Potentially impacts performance due to the overhead of coordination and consistency checks.
+* Saga-based implementations can lead to increased complexity in understanding the overall business process flow.
+
+## Related Patterns
+
+[Saga Pattern](https://java-design-patterns.com/patterns/saga/): Often discussed in tandem with the Commander pattern
+for distributed transactions, focusing on long-lived transactions with compensating actions.
 
 ## Credits
 
 * [Distributed Transactions: The Icebergs of Microservices](https://www.grahamlea.com/2016/08/distributed-transactions-microservices-icebergs/)
+* [Microservices Patterns: With examples in Java](https://amzn.to/4axjnYW)
+* [Designing Data-Intensive Applications: The Big Ideas Behind Reliable, Scalable, and Maintainable Systems](https://amzn.to/4axHwOV)
+* [Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions](https://amzn.to/4aATcRe)

commander/src/main/java/com/iluwatar/commander/Commander.java

@@ -121,7 +121,7 @@ void placeOrder(Order order) throws Exception {
     sendShippingRequest(order);
   }
 
-  private void sendShippingRequest(Order order) throws Exception {
+  private void sendShippingRequest(Order order) {
     var list = shippingService.exceptionsList;
     Retry.Operation op = (l) -> {
       if (!l.isEmpty()) {
@@ -233,7 +233,7 @@ private void sendPaymentRequest(Order order) {
       try {
         r.perform(list, order);
       } catch (Exception e1) {
-        e1.printStackTrace();
+        LOG.error("An exception occurred", e1);
       }
     });
     t.start();
@@ -282,7 +282,7 @@ private void updateQueue(QueueTask qt) {
       try {
         r.perform(list, qt);
       } catch (Exception e1) {
-        e1.printStackTrace();
+        LOG.error("An exception occurred", e1);
       }
     });
     t.start();
@@ -305,7 +305,7 @@ private void tryDoingTasksInQueue() { //commander controls operations done to qu
       try {
         r.perform(list, null);
       } catch (Exception e1) {
-        e1.printStackTrace();
+        LOG.error("An exception occurred", e1);
       }
     });
     t2.start();
@@ -324,12 +324,12 @@ private void tryDequeue() {
       };
       Retry.HandleErrorIssue<QueueTask> handleError = (o, err) -> {
       };
-      var r = new Retry<QueueTask>(op, handleError, numOfRetries, retryDuration,
+      var r = new Retry<>(op, handleError, numOfRetries, retryDuration,
           e -> DatabaseUnavailableException.class.isAssignableFrom(e.getClass()));
       try {
         r.perform(list, null);
       } catch (Exception e1) {
-        e1.printStackTrace();
+        LOG.error("An exception occurred", e1);
       }
     });
     t3.start();
@@ -351,7 +351,7 @@ private void sendSuccessMessage(Order order) {
       try {
         r.perform(list, order);
       } catch (Exception e1) {
-        e1.printStackTrace();
+        LOG.error("An exception occurred", e1);
       }
     });
     t.start();
@@ -409,7 +409,7 @@ private void sendPaymentFailureMessage(Order order) {
       try {
         r.perform(list, order);
       } catch (Exception e1) {
-        e1.printStackTrace();
+        LOG.error("An exception occurred", e1);
       }
     });
     t.start();
@@ -465,7 +465,7 @@ private void sendPaymentPossibleErrorMsg(Order order) {
       try {
         r.perform(list, order);
       } catch (Exception e1) {
-        e1.printStackTrace();
+        LOG.error("An exception occurred", e1);
       }
     });
     t.start();
@@ -537,7 +537,7 @@ private void employeeHandleIssue(Order order) {
       try {
         r.perform(list, order);
       } catch (Exception e1) {
-        e1.printStackTrace();
+        LOG.error("An exception occurred", e1);
       }
     });
     t.start();

commander/src/main/java/com/iluwatar/commander/Retry.java

@@ -94,7 +94,7 @@ public void perform(List<Exception> list, T obj) {
         this.errors.add(e);
         if (this.attempts.incrementAndGet() >= this.maxAttempts || !this.test.test(e)) {
           this.handleError.handleIssue(obj, e);
-          return; //return here...dont go further
+          return; //return here... don't go further
         }
         try {
           long testDelay =

commander/src/main/java/com/iluwatar/commander/employeehandle/EmployeeHandle.java

@@ -47,7 +47,7 @@ protected String updateDb(Object... parameters) throws DatabaseUnavailableExcept
     var o = (Order) parameters[0];
     if (database.get(o.id) == null) {
       database.add(o);
-      return o.id; //true rcvd - change addedToEmployeeHandle to true else dont do anything
+      return o.id; //true rcvd - change addedToEmployeeHandle to true else don't do anything
     }
     return null;
   }

commander/src/main/java/com/iluwatar/commander/messagingservice/MessagingDatabase.java

@@ -38,7 +38,7 @@ public class MessagingDatabase extends Database<MessageRequest> {
 
   @Override
   public MessageRequest add(MessageRequest r) {
-    return data.put(r.reqId, r);
+    return data.put(r.reqId(), r);
   }
 
   @Override

commander/src/main/java/com/iluwatar/commander/messagingservice/MessagingService.java

@@ -44,11 +44,7 @@ enum MessageToSend {
     PAYMENT_SUCCESSFUL
   }
 
-  @RequiredArgsConstructor
-  static class MessageRequest {
-    final String reqId;
-    final MessageToSend msg;
-  }
+  record MessageRequest(String reqId, MessageToSend msg) {}
 
   public MessagingService(MessagingDatabase db, Exception... exc) {
     super(db, exc);

commander/src/main/java/com/iluwatar/commander/paymentservice/PaymentService.java

@@ -51,7 +51,7 @@ public PaymentService(PaymentDatabase db, Exception... exc) {
    */
 
   public String receiveRequest(Object... parameters) throws DatabaseUnavailableException {
-    //it could also be sending a userid, payment details here or something, not added here
+    //it could also be sending an userid, payment details here or something, not added here
     var id = generateId();
     var req = new PaymentRequest(id, (float) parameters[0]);
     return updateDb(req);

commander/src/main/java/com/iluwatar/commander/queue/QueueDatabase.java

@@ -25,7 +25,6 @@
 package com.iluwatar.commander.queue;
 
 import com.iluwatar.commander.Database;
-import com.iluwatar.commander.exceptions.DatabaseUnavailableException;
 import com.iluwatar.commander.exceptions.IsEmptyException;
 import java.util.ArrayList;
 import java.util.List;

commander/src/test/java/com/iluwatar/commander/RetryTest.java

@@ -31,9 +31,13 @@
 import java.util.ArrayList;
 import java.util.List;
 import org.junit.jupiter.api.Test;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
 
 class RetryTest {
 
+  private static final Logger LOG = LoggerFactory.getLogger(RetryTest.class);
+
   @Test
   void performTest() {
     Retry.Operation op = (l) -> {
@@ -53,16 +57,16 @@ void performTest() {
     try {
       r1.perform(arr1, order);
     } catch (Exception e1) {
-      e1.printStackTrace();
+      LOG.error("An exception occurred", e1);
     }
     var arr2 = new ArrayList<>(List.of(new DatabaseUnavailableException(), new ItemUnavailableException()));
     try {
       r2.perform(arr2, order);
     } catch (Exception e1) {
-      e1.printStackTrace();
+      LOG.error("An exception occurred", e1);
     }
     //r1 stops at ItemUnavailableException, r2 retries because it encounters DatabaseUnavailableException
-    assertTrue(arr1.size() == 1 && arr2.size() == 0);
+    assertTrue(arr1.size() == 1 && arr2.isEmpty());
   }
 
 }