From f8e71ec572adc40968b88cf2e7ac7e92f36fb1c4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ilkka=20Sepp=C3=A4l=C3=A4?= Date: Sat, 30 Mar 2024 13:19:34 +0200 Subject: [PATCH 1/2] collection pipeline docs + refactoring --- collection-pipeline/README.md | 136 ++++++++++++++++-- .../com/iluwatar/collectionpipeline/Car.java | 17 +-- .../FunctionalProgramming.java | 14 +- .../ImperativeProgramming.java | 28 ++-- .../iluwatar/collectionpipeline/Person.java | 10 +- 5 files changed, 148 insertions(+), 57 deletions(-) diff --git a/collection-pipeline/README.md b/collection-pipeline/README.md index 3ae285accb69..aab323e0ec14 100644 --- a/collection-pipeline/README.md +++ b/collection-pipeline/README.md @@ -3,25 +3,145 @@ title: Collection Pipeline category: Functional language: en tag: - - Reactive + - Reactive + - Data processing --- ## Intent -Collection Pipeline introduces Function Composition and Collection Pipeline, two functional-style patterns that you can combine to iterate collections in your code. -In functional programming, it's common to sequence complex operations through a series of smaller modular functions or operations. The series is called a composition of functions, or a function composition. When a collection of data flows through a function composition, it becomes a collection pipeline. Function Composition and Collection Pipeline are two design patterns frequently used in functional-style programming. + +The Collection Pipeline design pattern is intended to process collections of data by chaining together operations in a +sequence where the output of one operation is the input for the next. It promotes a declarative approach to handling +collections, focusing on what should be done rather than how. + +## Explanation + +Real-world example + +> Imagine you're in a large library filled with books, and you're tasked with finding all the science fiction books +> published after 2000, then arranging them by author name in alphabetical order, and finally picking out the top 5 based +> on their popularity or ratings. + +In plain words + +> The Collection Pipeline pattern involves processing data by passing it through a series of operations, each +> transforming the data in sequence, much like an assembly line in a factory. + +Wikipedia says + +> In software engineering, a pipeline consists of a chain of processing elements (processes, threads, coroutines, +> functions, etc.), arranged so that the output of each element is the input of the next; the name is by analogy to a +> physical pipeline. Usually some amount of buffering is provided between consecutive elements. The information that flows +> in these pipelines is often a stream of records, bytes, or bits, and the elements of a pipeline may be called filters; +> this is also called the pipe(s) and filters design pattern. Connecting elements into a pipeline is analogous to function +> composition. + +**Programmatic Example** + +The Collection Pipeline pattern is implemented in this code example by using Java's Stream API to perform a series of +transformations on a collection of Car objects. The transformations are chained together to form a pipeline. Here's a +breakdown of how it's done: + +1. Creation of Cars: A list of Car objects is created using the `CarFactory.createCars()` method. + +`var cars = CarFactory.createCars();` + +2. Filtering and Transforming: The `FunctionalProgramming.getModelsAfter2000(cars)` method filters the cars to only + include those made after the year 2000, and then transforms the filtered cars into a list of their model names. + +`var modelsFunctional = FunctionalProgramming.getModelsAfter2000(cars);` + +In the `getModelsAfter2000` method, the pipeline is created as follows: + +```java +public static List getModelsAfter2000(List cars){ + return cars.stream().filter(car->car.getYear()>2000) + .sorted(comparing(Car::getYear)) + .map(Car::getModel) + .collect(toList()); + } +``` + +3. Grouping: The `FunctionalProgramming.getGroupingOfCarsByCategory(cars)` method groups the cars by their category. + +`var groupingByCategoryFunctional = FunctionalProgramming.getGroupingOfCarsByCategory(cars);` + +In the getGroupingOfCarsByCategory method, the pipeline is created as follows: + +```java +public static Map>getGroupingOfCarsByCategory(List cars){ + return cars.stream().collect(groupingBy(Car::getCategory)); + } +``` + +4. Filtering, Sorting and Transforming: The `FunctionalProgramming.getSedanCarsOwnedSortedByDate(List.of(john))` method + filters the cars owned by a person to only include sedans, sorts them by date, and then transforms the sorted cars + into a list of Car objects. + +`var sedansOwnedFunctional = FunctionalProgramming.getSedanCarsOwnedSortedByDate(List.of(john));` + +In the `getSedanCarsOwnedSortedByDate` method, the pipeline is created as follows: + +```java +public static List getSedanCarsOwnedSortedByDate(List persons){ + return persons.stream().flatMap(person->person.getCars().stream()) + .filter(car->Category.SEDAN.equals(car.getCategory())) + .sorted(comparing(Car::getDate)) + .collect(toList()); + } +``` + +In each of these methods, the Collection Pipeline pattern is used to perform a series of operations on the collection of +cars in a declarative manner, which improves readability and maintainability. ## Class diagram + ![alt text](./etc/collection-pipeline.png "Collection Pipeline") ## Applicability -Use the Collection Pipeline pattern when -* When you want to perform a sequence of operations where one operation's collected output is fed into the next -* When you use a lot of statements in your code -* When you use a lot of loops in your code +This pattern is applicable in scenarios involving bulk data operations such as filtering, mapping, sorting, or reducing +collections. It's particularly useful in data analysis, transformation tasks, and where a sequence of operations needs +to be applied to each element of a collection. + +## Known Uses + +* LINQ in .NET +* Stream API in Java 8+ +* Collections in modern functional languages (e.g., Haskell, Scala) +* Database query builders and ORM frameworks + +## Consequences + +Benefits: + +* Readability: The code is more readable and declarative, making it easier to understand the sequence of operations. +* Maintainability: Easier to modify or extend the pipeline with additional operations. +* Reusability: Common operations can be abstracted into reusable functions. +* Lazy Evaluation: Some implementations allow for operations to be lazily evaluated, improving performance. + +Trade-offs: + +* Performance Overhead: Chaining multiple operations can introduce overhead compared to traditional loops, especially + for short pipelines or very large collections. +* Debugging Difficulty: Debugging a chain of operations might be more challenging due to the lack of intermediate + variables. +* Limited to Collections: Primarily focused on collections, and its utility might be limited outside of collection + processing. + +## Related Patterns + +* [Builder](https://java-design-patterns.com/patterns/builder/): Similar fluent interface style but used for object + construction. +* [Chain of Responsibility](https://java-design-patterns.com/patterns/chain-of-responsibility/): Conceptually similar in + chaining handlers, but applied to object requests rather than data collection processing. +* [Strategy](https://java-design-patterns.com/patterns/strategy/): Can be used within a pipeline stage to encapsulate + different algorithms that can be selected at runtime. ## Credits * [Function composition and the Collection Pipeline pattern](https://www.ibm.com/developerworks/library/j-java8idioms2/index.html) -* [Martin Fowler](https://martinfowler.com/articles/collection-pipeline/) +* [Collection Pipeline described by Martin Fowler](https://martinfowler.com/articles/collection-pipeline/) * [Java8 Streams](https://docs.oracle.com/javase/8/docs/api/java/util/stream/package-summary.html) +* [Refactoring: Improving the Design of Existing Code](https://amzn.to/3VDMWDO) +* [Functional Programming in Scala](https://amzn.to/4cEo6K2) +* [Java 8 in Action: Lambdas, Streams, and functional-style programming](https://amzn.to/3THp4wy) diff --git a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Car.java b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Car.java index 97e13231b24f..2cfeb963ba49 100644 --- a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Car.java +++ b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Car.java @@ -22,22 +22,11 @@ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN * THE SOFTWARE. */ -package com.iluwatar.collectionpipeline; -import lombok.EqualsAndHashCode; -import lombok.Getter; -import lombok.RequiredArgsConstructor; +package com.iluwatar.collectionpipeline; /** * A Car class that has the properties of make, model, year and category. */ -@Getter -@EqualsAndHashCode -@RequiredArgsConstructor -public class Car { - private final String make; - private final String model; - private final int year; - private final Category category; - -} \ No newline at end of file +public record Car(String make, String model, int year, Category category) { +} diff --git a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/FunctionalProgramming.java b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/FunctionalProgramming.java index adf0fda4bda2..20bf77d29a2f 100644 --- a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/FunctionalProgramming.java +++ b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/FunctionalProgramming.java @@ -22,6 +22,7 @@ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN * THE SOFTWARE. */ + package com.iluwatar.collectionpipeline; import java.util.Comparator; @@ -54,9 +55,8 @@ private FunctionalProgramming() { * @return {@link List} of {@link String} representing models built after year 2000 */ public static List getModelsAfter2000(List cars) { - return cars.stream().filter(car -> car.getYear() > 2000) - .sorted(Comparator.comparing(Car::getYear)) - .map(Car::getModel).toList(); + return cars.stream().filter(car -> car.year() > 2000).sorted(Comparator.comparing(Car::year)) + .map(Car::model).toList(); } /** @@ -66,7 +66,7 @@ public static List getModelsAfter2000(List cars) { * @return {@link Map} with category as key and cars belonging to that category as value */ public static Map> getGroupingOfCarsByCategory(List cars) { - return cars.stream().collect(Collectors.groupingBy(Car::getCategory)); + return cars.stream().collect(Collectors.groupingBy(Car::category)); } /** @@ -76,8 +76,8 @@ public static Map> getGroupingOfCarsByCategory(List car * @return {@link List} of {@link Car} to belonging to the group */ public static List getSedanCarsOwnedSortedByDate(List persons) { - return persons.stream().map(Person::getCars).flatMap(List::stream) - .filter(car -> Category.SEDAN.equals(car.getCategory())) - .sorted(Comparator.comparing(Car::getYear)).toList(); + return persons.stream().map(Person::cars).flatMap(List::stream) + .filter(car -> Category.SEDAN.equals(car.category())) + .sorted(Comparator.comparing(Car::year)).toList(); } } diff --git a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java index e3359a389b0d..47e8f657ccc9 100644 --- a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java +++ b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java @@ -61,21 +61,16 @@ public static List getModelsAfter2000(List cars) { List carsSortedByYear = new ArrayList<>(); for (Car car : cars) { - if (car.getYear() > 2000) { + if (car.year() > 2000) { carsSortedByYear.add(car); } } - Collections.sort(carsSortedByYear, new Comparator() { - @Override - public int compare(Car car1, Car car2) { - return car1.getYear() - car2.getYear(); - } - }); + carsSortedByYear.sort(Comparator.comparingInt(Car::year)); List models = new ArrayList<>(); for (Car car : carsSortedByYear) { - models.add(car.getModel()); + models.add(car.model()); } return models; @@ -90,12 +85,12 @@ public int compare(Car car1, Car car2) { public static Map> getGroupingOfCarsByCategory(List cars) { Map> groupingByCategory = new HashMap<>(); for (Car car : cars) { - if (groupingByCategory.containsKey(car.getCategory())) { - groupingByCategory.get(car.getCategory()).add(car); + if (groupingByCategory.containsKey(car.category())) { + groupingByCategory.get(car.category()).add(car); } else { List categoryCars = new ArrayList<>(); categoryCars.add(car); - groupingByCategory.put(car.getCategory(), categoryCars); + groupingByCategory.put(car.category(), categoryCars); } } return groupingByCategory; @@ -111,22 +106,17 @@ public static Map> getGroupingOfCarsByCategory(List car public static List getSedanCarsOwnedSortedByDate(List persons) { List cars = new ArrayList<>(); for (Person person : persons) { - cars.addAll(person.getCars()); + cars.addAll(person.cars()); } List sedanCars = new ArrayList<>(); for (Car car : cars) { - if (Category.SEDAN.equals(car.getCategory())) { + if (Category.SEDAN.equals(car.category())) { sedanCars.add(car); } } - sedanCars.sort(new Comparator() { - @Override - public int compare(Car o1, Car o2) { - return o1.getYear() - o2.getYear(); - } - }); + sedanCars.sort(Comparator.comparingInt(Car::year)); return sedanCars; } diff --git a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Person.java b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Person.java index 80b6a8bf0303..992596125423 100644 --- a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Person.java +++ b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/Person.java @@ -25,16 +25,8 @@ package com.iluwatar.collectionpipeline; import java.util.List; -import lombok.Getter; -import lombok.RequiredArgsConstructor; /** * A Person class that has the list of cars that the person owns and use. */ -@Getter -@RequiredArgsConstructor -public class Person { - - private final List cars; - -} \ No newline at end of file +public record Person(List cars) {} From 9f4a801c33637322a883491fa3084ffed120a0ae Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ilkka=20Sepp=C3=A4l=C3=A4?= Date: Sat, 30 Mar 2024 13:28:57 +0200 Subject: [PATCH 2/2] restore imperative programming code --- .../collectionpipeline/ImperativeProgramming.java | 14 ++++++++++++-- 1 file changed, 12 insertions(+), 2 deletions(-) diff --git a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java index 47e8f657ccc9..2a85334c6643 100644 --- a/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java +++ b/collection-pipeline/src/main/java/com/iluwatar/collectionpipeline/ImperativeProgramming.java @@ -66,7 +66,12 @@ public static List getModelsAfter2000(List cars) { } } - carsSortedByYear.sort(Comparator.comparingInt(Car::year)); + Collections.sort(carsSortedByYear, new Comparator() { + @Override + public int compare(Car car1, Car car2) { + return car1.year() - car2.year(); + } + }); List models = new ArrayList<>(); for (Car car : carsSortedByYear) { @@ -116,7 +121,12 @@ public static List getSedanCarsOwnedSortedByDate(List persons) { } } - sedanCars.sort(Comparator.comparingInt(Car::year)); + sedanCars.sort(new Comparator() { + @Override + public int compare(Car o1, Car o2) { + return o1.year() - o2.year(); + } + }); return sedanCars; }