Collection pipeline explanation

collection-pipeline/README.md

@@ -3,25 +3,145 @@ title: Collection Pipeline
 category: Functional
 language: en
 tag:
- - Reactive
+    - Reactive
+    - Data processing
 ---
 
 ## Intent
-Collection Pipeline introduces Function Composition and Collection Pipeline, two functional-style patterns that you can combine to iterate collections in your code. 
-In functional programming, it's common to sequence complex operations through a series of smaller modular functions or operations. The series is called a composition of functions, or a function composition. When a collection of data flows through a function composition, it becomes a collection pipeline. Function Composition and Collection Pipeline are two design patterns frequently used in functional-style programming.
+
+The Collection Pipeline design pattern is intended to process collections of data by chaining together operations in a
+sequence where the output of one operation is the input for the next. It promotes a declarative approach to handling
+collections, focusing on what should be done rather than how.
+
+## Explanation
+
+Real-world example
+
+> Imagine you're in a large library filled with books, and you're tasked with finding all the science fiction books
+> published after 2000, then arranging them by author name in alphabetical order, and finally picking out the top 5 based
+> on their popularity or ratings.
+
+In plain words
+
+> The Collection Pipeline pattern involves processing data by passing it through a series of operations, each
+> transforming the data in sequence, much like an assembly line in a factory.
+
+Wikipedia says
+
+> In software engineering, a pipeline consists of a chain of processing elements (processes, threads, coroutines,
+> functions, etc.), arranged so that the output of each element is the input of the next; the name is by analogy to a
+> physical pipeline. Usually some amount of buffering is provided between consecutive elements. The information that flows
+> in these pipelines is often a stream of records, bytes, or bits, and the elements of a pipeline may be called filters;
+> this is also called the pipe(s) and filters design pattern. Connecting elements into a pipeline is analogous to function
+> composition.
+
+**Programmatic Example**
+
+The Collection Pipeline pattern is implemented in this code example by using Java's Stream API to perform a series of
+transformations on a collection of Car objects. The transformations are chained together to form a pipeline. Here's a
+breakdown of how it's done:
+
+1. Creation of Cars: A list of Car objects is created using the `CarFactory.createCars()` method.
+
+`var cars = CarFactory.createCars();`
+
+2. Filtering and Transforming: The `FunctionalProgramming.getModelsAfter2000(cars)` method filters the cars to only
+   include those made after the year 2000, and then transforms the filtered cars into a list of their model names.
+
+`var modelsFunctional = FunctionalProgramming.getModelsAfter2000(cars);`
+
+In the `getModelsAfter2000` method, the pipeline is created as follows:
+
+```java
+public static List<String> getModelsAfter2000(List<Car> cars){
+        return cars.stream().filter(car->car.getYear()>2000)
+        .sorted(comparing(Car::getYear))
+        .map(Car::getModel)
+        .collect(toList());
+        }
+```
+
+3. Grouping: The `FunctionalProgramming.getGroupingOfCarsByCategory(cars)` method groups the cars by their category.
+
+`var groupingByCategoryFunctional = FunctionalProgramming.getGroupingOfCarsByCategory(cars);`
+
+In the getGroupingOfCarsByCategory method, the pipeline is created as follows:
+
+```java
+public static Map<Category, List<Car>>getGroupingOfCarsByCategory(List<Car> cars){
+        return cars.stream().collect(groupingBy(Car::getCategory));
+        }
+```
+
+4. Filtering, Sorting and Transforming: The `FunctionalProgramming.getSedanCarsOwnedSortedByDate(List.of(john))` method
+   filters the cars owned by a person to only include sedans, sorts them by date, and then transforms the sorted cars
+   into a list of Car objects.
+
+`var sedansOwnedFunctional = FunctionalProgramming.getSedanCarsOwnedSortedByDate(List.of(john));`
+
+In the `getSedanCarsOwnedSortedByDate` method, the pipeline is created as follows:
+
+```java
+public static List<Car> getSedanCarsOwnedSortedByDate(List<Person> persons){
+        return persons.stream().flatMap(person->person.getCars().stream())
+        .filter(car->Category.SEDAN.equals(car.getCategory()))
+        .sorted(comparing(Car::getDate))
+        .collect(toList());
+        }
+```
+
+In each of these methods, the Collection Pipeline pattern is used to perform a series of operations on the collection of
+cars in a declarative manner, which improves readability and maintainability.
 
 ## Class diagram
+
 ![alt text](./etc/collection-pipeline.png "Collection Pipeline")
 
 ## Applicability
-Use the Collection Pipeline pattern when
 
-* When you want to perform a sequence of operations where one operation's collected output is fed into the next
-* When you use a lot of statements in your code
-* When you use a lot of loops in your code
+This pattern is applicable in scenarios involving bulk data operations such as filtering, mapping, sorting, or reducing
+collections. It's particularly useful in data analysis, transformation tasks, and where a sequence of operations needs
+to be applied to each element of a collection.
+
+## Known Uses
+
+* LINQ in .NET
+* Stream API in Java 8+
+* Collections in modern functional languages (e.g., Haskell, Scala)
+* Database query builders and ORM frameworks
+
+## Consequences
+
+Benefits:
+
+* Readability: The code is more readable and declarative, making it easier to understand the sequence of operations.
+* Maintainability: Easier to modify or extend the pipeline with additional operations.
+* Reusability: Common operations can be abstracted into reusable functions.
+* Lazy Evaluation: Some implementations allow for operations to be lazily evaluated, improving performance.
+
+Trade-offs:
+
+* Performance Overhead: Chaining multiple operations can introduce overhead compared to traditional loops, especially
+  for short pipelines or very large collections.
+* Debugging Difficulty: Debugging a chain of operations might be more challenging due to the lack of intermediate
+  variables.
+* Limited to Collections: Primarily focused on collections, and its utility might be limited outside of collection
+  processing.
+
+## Related Patterns
+
+* [Builder](https://java-design-patterns.com/patterns/builder/): Similar fluent interface style but used for object
+  construction.
+* [Chain of Responsibility](https://java-design-patterns.com/patterns/chain-of-responsibility/): Conceptually similar in
+  chaining handlers, but applied to object requests rather than data collection processing.
+* [Strategy](https://java-design-patterns.com/patterns/strategy/): Can be used within a pipeline stage to encapsulate
+  different algorithms that can be selected at runtime.
 
 ## Credits
 
 * [Function composition and the Collection Pipeline pattern](https://www.ibm.com/developerworks/library/j-java8idioms2/index.html)
-* [Martin Fowler](https://martinfowler.com/articles/collection-pipeline/)
+* [Collection Pipeline described by Martin Fowler](https://martinfowler.com/articles/collection-pipeline/)
 * [Java8 Streams](https://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html)
+* [Refactoring: Improving the Design of Existing Code](https://amzn.to/3VDMWDO)
+* [Functional Programming in Scala](https://amzn.to/4cEo6K2)
+* [Java 8 in Action: Lambdas, Streams, and functional-style programming](https://amzn.to/3THp4wy)

collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Car.java

@@ -22,22 +22,11 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  */
-package com.iluwatar.collectionpipeline;
 
-import lombok.EqualsAndHashCode;
-import lombok.Getter;
-import lombok.RequiredArgsConstructor;
+package com.iluwatar.collectionpipeline;
 
 /**
  * A Car class that has the properties of make, model, year and category.
  */
-@Getter
-@EqualsAndHashCode
-@RequiredArgsConstructor
-public class Car {
-  private final String make;
-  private final String model;
-  private final int year;
-  private final Category category;
-
-}
+public record Car(String make, String model, int year, Category category) {
+}

collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/FunctionalProgramming.java

@@ -22,6 +22,7 @@
  * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
  * THE SOFTWARE.
  */
+
 package com.iluwatar.collectionpipeline;
 
 import java.util.Comparator;
@@ -54,9 +55,8 @@ private FunctionalProgramming() {
    * @return {@link List} of {@link String} representing models built after year 2000
    */
   public static List<String> getModelsAfter2000(List<Car> cars) {
-    return cars.stream().filter(car -> car.getYear() > 2000)
-        .sorted(Comparator.comparing(Car::getYear))
-        .map(Car::getModel).toList();
+    return cars.stream().filter(car -> car.year() > 2000).sorted(Comparator.comparing(Car::year))
+        .map(Car::model).toList();
   }
 
   /**
@@ -66,7 +66,7 @@ public static List<String> getModelsAfter2000(List<Car> cars) {
    * @return {@link Map} with category as key and cars belonging to that category as value
    */
   public static Map<Category, List<Car>> getGroupingOfCarsByCategory(List<Car> cars) {
-    return cars.stream().collect(Collectors.groupingBy(Car::getCategory));
+    return cars.stream().collect(Collectors.groupingBy(Car::category));
   }
 
   /**
@@ -76,8 +76,8 @@ public static Map<Category, List<Car>> getGroupingOfCarsByCategory(List<Car> car
    * @return {@link List} of {@link Car} to belonging to the group
    */
   public static List<Car> getSedanCarsOwnedSortedByDate(List<Person> persons) {
-    return persons.stream().map(Person::getCars).flatMap(List::stream)
-        .filter(car -> Category.SEDAN.equals(car.getCategory()))
-        .sorted(Comparator.comparing(Car::getYear)).toList();
+    return persons.stream().map(Person::cars).flatMap(List::stream)
+        .filter(car -> Category.SEDAN.equals(car.category()))
+        .sorted(Comparator.comparing(Car::year)).toList();
   }
 }

collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java

@@ -61,21 +61,21 @@ public static List<String> getModelsAfter2000(List<Car> cars) {
     List<Car> carsSortedByYear = new ArrayList<>();
 
     for (Car car : cars) {
-      if (car.getYear() > 2000) {
+      if (car.year() > 2000) {
         carsSortedByYear.add(car);
       }
     }
 
     Collections.sort(carsSortedByYear, new Comparator<Car>() {
       @Override
       public int compare(Car car1, Car car2) {
-        return car1.getYear() - car2.getYear();
+        return car1.year() - car2.year();
       }
     });
 
     List<String> models = new ArrayList<>();
     for (Car car : carsSortedByYear) {
-      models.add(car.getModel());
+      models.add(car.model());
     }
 
     return models;
@@ -90,12 +90,12 @@ public int compare(Car car1, Car car2) {
   public static Map<Category, List<Car>> getGroupingOfCarsByCategory(List<Car> cars) {
     Map<Category, List<Car>> groupingByCategory = new HashMap<>();
     for (Car car : cars) {
-      if (groupingByCategory.containsKey(car.getCategory())) {
-        groupingByCategory.get(car.getCategory()).add(car);
+      if (groupingByCategory.containsKey(car.category())) {
+        groupingByCategory.get(car.category()).add(car);
       } else {
         List<Car> categoryCars = new ArrayList<>();
         categoryCars.add(car);
-        groupingByCategory.put(car.getCategory(), categoryCars);
+        groupingByCategory.put(car.category(), categoryCars);
       }
     }
     return groupingByCategory;
@@ -111,20 +111,20 @@ public static Map<Category, List<Car>> getGroupingOfCarsByCategory(List<Car> car
   public static List<Car> getSedanCarsOwnedSortedByDate(List<Person> persons) {
     List<Car> cars = new ArrayList<>();
     for (Person person : persons) {
-      cars.addAll(person.getCars());
+      cars.addAll(person.cars());
     }
 
     List<Car> sedanCars = new ArrayList<>();
     for (Car car : cars) {
-      if (Category.SEDAN.equals(car.getCategory())) {
+      if (Category.SEDAN.equals(car.category())) {
         sedanCars.add(car);
       }
     }
 
     sedanCars.sort(new Comparator<Car>() {
       @Override
       public int compare(Car o1, Car o2) {
-        return o1.getYear() - o2.getYear();
+        return o1.year() - o2.year();
       }
     });
 

collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Person.java

@@ -25,16 +25,8 @@
 package com.iluwatar.collectionpipeline;
 
 import java.util.List;
-import lombok.Getter;
-import lombok.RequiredArgsConstructor;
 
 /**
  * A Person class that has the list of cars that the person owns and use.
  */
-@Getter
-@RequiredArgsConstructor
-public class Person {
-
-  private final List<Car> cars;
-
-}
+public record Person(List<Car> cars) {}