diff --git a/microservices-bulkhead/README.md b/microservices-bulkhead/README.md new file mode 100644 index 000000000000..be7851c77598 --- /dev/null +++ b/microservices-bulkhead/README.md @@ -0,0 +1,192 @@ +--- +title: "Bulkhead Pattern in Java: Isolating Resources for Resilient Microservices" +shortTitle: Bulkhead +description: "Learn how the Bulkhead pattern in Java isolates critical system resources to prevent cascade failures in microservices. Includes real-world examples, code demonstrations, and best practices for building resilient distributed systems." +category: Resilience +language: en +tag: + - Resilience + - Fault tolerance + - Microservices + - Performance + - Scalability + - Thread management +--- + +## Also known as + +* Resource Isolation Pattern +* Partition Pattern + +## Intent of Bulkhead Design Pattern + +The Bulkhead pattern isolates critical system resources for each service or component to prevent failures or heavy load in one part of the system from cascading and degrading the entire application. By partitioning resources—often via separate thread pools or connection pools—the system ensures other services remain operational even if one service becomes overloaded or fails. + +## Detailed Explanation of Bulkhead Pattern with Real-World Examples + +Real-world example + +> Consider a modern cruise ship with multiple watertight compartments (bulkheads). If one compartment is breached and starts flooding, the bulkheads prevent water from spreading to other compartments, keeping the ship afloat. Similarly, in software systems, the Bulkhead pattern creates isolated resource pools for different services. If one service experiences issues (like a slow external API or heavy load), it only affects its dedicated resources, while other services continue operating normally with their own resource pools. + +In plain words + +> The Bulkhead pattern partitions system resources into isolated pools so that failures in one area don't consume all available resources and bring down the entire system. + +## Programmatic Example of Bulkhead Pattern in Java + +The Bulkhead pattern implementation demonstrates resource isolation using dedicated thread pools for different services. Here we have a `UserService` handling critical user requests and a `BackgroundService` handling non-critical tasks. + +First, let's look at the base `BulkheadService` class that provides resource isolation: + +``` +public abstract class BulkheadService { +private static final Logger LOGGER = LoggerFactory.getLogger(BulkheadService.class); + +protected final ThreadPoolExecutor executor; +protected final String serviceName; + +protected BulkheadService(String serviceName, int maxPoolSize, int queueCapacity) { +this.serviceName = serviceName; + + // Create thread pool with bounded queue for resource isolation + this.executor = new ThreadPoolExecutor( + maxPoolSize, + maxPoolSize, + 60L, + TimeUnit.SECONDS, + new ArrayBlockingQueue<>(queueCapacity), + new ThreadPoolExecutor.AbortPolicy() // fail-fast when full + ); + + LOGGER.info("Created {} with {} threads and queue capacity {}", + serviceName, maxPoolSize, queueCapacity); +} + +public boolean submitTask(Task task) { +try { +executor.execute(() -> processTask(task)); +LOGGER.info("[{}] Task '{}' submitted successfully", serviceName, task.getName()); +return true; +} catch (RejectedExecutionException e) { +LOGGER.warn("[{}] Task '{}' REJECTED - bulkhead is full", serviceName, task.getName()); +handleRejectedTask(task); +return false; +} +} +} +``` + +The `UserService` handles critical user-facing requests with dedicated resources: + +``` +public class UserService extends BulkheadService { +private static final int DEFAULT_QUEUE_CAPACITY = 10; + +public UserService(int maxThreads) { +super("UserService", maxThreads, DEFAULT_QUEUE_CAPACITY); +} +} +``` + +The `BackgroundService` handles non-critical background tasks with its own isolated resources: + +``` +public class BackgroundService extends BulkheadService { +private static final int DEFAULT_QUEUE_CAPACITY = 20; + +public BackgroundService(int maxThreads) { +super("BackgroundService", maxThreads, DEFAULT_QUEUE_CAPACITY); +} +} +``` + +Here's the demonstration showing how the Bulkhead pattern prevents cascade failures: + +``` +public class App { +public static void main(String[] args) { +BulkheadService userService = new UserService(3); +BulkheadService backgroundService = new BackgroundService(2); + + // Overload background service with many tasks + for (int i = 1; i <= 10; i++) { + Task task = new Task("Heavy-Background-Job-" + i, TaskType.BACKGROUND_PROCESSING, 2000); + backgroundService.submitTask(task); + } + + // User service remains responsive despite background service overload + for (int i = 1; i <= 3; i++) { + Task task = new Task("Critical-User-Request-" + i, TaskType.USER_REQUEST, 300); + boolean accepted = userService.submitTask(task); + LOGGER.info("User request {} accepted: {}", i, accepted); + } +} +} +``` + +Program output: + +``` +[BackgroundService] Task 'Heavy-Background-Job-1' submitted successfully +[BackgroundService] Task 'Heavy-Background-Job-2' submitted successfully +[BackgroundService] Task 'Heavy-Background-Job-3' REJECTED - bulkhead is full +... +[UserService] Task 'Critical-User-Request-1' submitted successfully +[UserService] Task 'Critical-User-Request-2' submitted successfully +[UserService] Task 'Critical-User-Request-3' submitted successfully +User request 1 accepted: true +User request 2 accepted: true +User request 3 accepted: true +``` + +The output demonstrates that even when the background service is overloaded and rejecting tasks, the user service continues to accept and process requests successfully due to resource isolation. + +## When to Use the Bulkhead Pattern in Java + +* When building microservices architectures where service failures should not cascade +* When different operations have varying criticality levels (e.g., user-facing vs. background tasks) +* When external dependencies (databases, APIs) might become slow or unavailable +* When you need to guarantee minimum service levels for critical operations +* In high-throughput systems where resource exhaustion in one area could impact other services + +## Real-World Applications of Bulkhead Pattern in Java + +* Netflix's Hystrix library implements bulkheads using thread pool isolation +* Resilience4j provides bulkhead implementations for Java applications +* AWS Lambda functions run in isolated execution environments (bulkheads) +* Kubernetes resource limits and quotas implement bulkhead principles +* Database connection pools per service in microservices architectures + +## Benefits and Trade-offs of Bulkhead Pattern + +Benefits: + +* Prevents cascade failures across services +* Improves system resilience and availability +* Provides predictable degradation under load +* Enables independent scaling of different services +* Facilitates easier capacity planning and monitoring + +Trade-offs: + +* Increased resource overhead (multiple thread pools) +* More complex configuration and tuning +* Potential for resource underutilization if pools are too large +* Requires careful capacity planning for each bulkhead +* May increase overall latency due to queuing + +## Related Java Design Patterns + +* [Circuit Breaker](https://java-design-patterns.com/patterns/circuit-breaker/): Often used together with Bulkhead; Circuit Breaker stops calling failing services while Bulkhead limits resources +* [Retry](https://java-design-patterns.com/patterns/retry/): Can be combined with Bulkhead for transient failure handling +* [Throttling](https://java-design-patterns.com/patterns/throttling/): Similar goal of resource management but focuses on rate limiting rather than isolation +* [Load Balancer](https://java-design-patterns.com/patterns/load-balancer/): Works at request distribution level while Bulkhead works at resource isolation level + +## References and Credits + +* [Release It!: Design and Deploy Production-Ready Software](https://amzn.to/3Uul4kF) - Michael T. Nygard +* [Microservices Patterns: With examples in Java](https://amzn.to/3UyWD5O) - Chris Richardson +* [Building Microservices: Designing Fine-Grained Systems](https://amzn.to/3RYRz96) - Sam Newman +* [Resilience4j Documentation - Bulkhead](https://resilience4j.readme.io/docs/bulkhead) +* [Microsoft Azure Architecture - Bulkhead Pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/bulkhead) +* [Microservices.io - Bulkhead Pattern](https://microservices.io/patterns/reliability/bulkhead.html) \ No newline at end of file diff --git a/microservices-bulkhead/pom.xml b/microservices-bulkhead/pom.xml new file mode 100644 index 000000000000..0d11453a0527 --- /dev/null +++ b/microservices-bulkhead/pom.xml @@ -0,0 +1,144 @@ + + + + 4.0.0 + + + com.iluwatar + java-design-patterns + 1.26.0-SNAPSHOT + + + bulkhead + jar + + Bulkhead + + The Bulkhead pattern isolates critical system resources for each service or component + to prevent failures or heavy load in one part from cascading and degrading the entire system. + By partitioning resources via separate thread pools, the system ensures other services + remain operational even if one service becomes overloaded or fails. + + + + UTF-8 + + + + + + org.slf4j + slf4j-api + + + + + ch.qos.logback + logback-classic + runtime + + + + + org.projectlombok + lombok + provided + + + + + org.junit.jupiter + junit-jupiter-engine + test + + + + + org.junit.jupiter + junit-jupiter-params + test + + + + + org.mockito + mockito-core + test + + + + + org.assertj + assertj-core + test + + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + + + + + org.apache.maven.plugins + maven-surefire-plugin + + + + + com.diffplug.spotless + spotless-maven-plugin + + + + + org.apache.maven.plugins + maven-checkstyle-plugin + + + + + org.apache.maven.plugins + maven-pmd-plugin + + + + + com.github.spotbugs + spotbugs-maven-plugin + + + + + diff --git a/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java new file mode 100644 index 000000000000..194a52f8e819 --- /dev/null +++ b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/App.java @@ -0,0 +1,137 @@ +package com.iluwatar.bulkhead; + +import java.util.concurrent.TimeUnit; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * The Bulkhead pattern isolates critical system resources for each service or component + * to prevent failures or heavy load in one part from cascading and degrading the entire system. + * + *

In this example, we demonstrate resource isolation using separate thread pools for + * different services. The {@link UserService} handles user-facing requests while + * {@link BackgroundService} handles background tasks. Each service has its own dedicated + * thread pool (bulkhead), ensuring that if one service becomes overloaded, the other + * continues to function normally. + * + *

Key concepts demonstrated: + *

+ */ +public class App { + + private static final Logger LOGGER = LoggerFactory.getLogger(App.class); + + /** + * Program entry point. + * + * @param args command line arguments + */ + public static void main(String[] args) { + LOGGER.info("Starting Bulkhead Pattern demonstration"); + + // Create bulkhead services with different thread pool sizes + BulkheadService userService = new UserService(3); // 3 threads for user requests + BulkheadService backgroundService = new BackgroundService(2); // 2 threads for background tasks + + try { + // Scenario 1: Normal operation - both services handle requests successfully + LOGGER.info("\n=== Scenario 1: Normal Operation ==="); + demonstrateNormalOperation(userService, backgroundService); + + Thread.sleep(2000); // Wait for tasks to complete + + // Scenario 2: Overload background service - user service should remain unaffected + LOGGER.info("\n=== Scenario 2: Background Service Overload ==="); + demonstrateServiceOverload(userService, backgroundService); + + Thread.sleep(3000); // Wait for demonstration to complete + + // Scenario 3: Show resource isolation + LOGGER.info("\n=== Scenario 3: Resource Isolation ==="); + demonstrateResourceIsolation(userService, backgroundService); + + } catch (InterruptedException e) { + LOGGER.error("Demonstration interrupted", e); + Thread.currentThread().interrupt(); + } finally { + // Cleanup + LOGGER.info("\nShutting down services"); + userService.shutdown(); + backgroundService.shutdown(); + LOGGER.info("Bulkhead Pattern demonstration completed"); + } + } + + /** + * Demonstrates normal operation where both services handle requests successfully. + * + * @param userService the user-facing service + * @param backgroundService the background processing service + */ + private static void demonstrateNormalOperation( + BulkheadService userService, + BulkheadService backgroundService) { + + LOGGER.info("Submitting normal workload to both services"); + + // Submit tasks to user service + for (int i = 1; i <= 3; i++) { + Task task = new Task("User-Request-" + i, TaskType.USER_REQUEST, 500); + userService.submitTask(task); + } + + // Submit tasks to background service + for (int i = 1; i <= 2; i++) { + Task task = new Task("Background-Job-" + i, TaskType.BACKGROUND_PROCESSING, 700); + backgroundService.submitTask(task); + } + } + + /** + * Demonstrates overloading the background service while user service remains operational. + * This shows how the bulkhead pattern prevents cascade failures. + * + * @param userService the user-facing service + * @param backgroundService the background processing service + */ + private static void demonstrateServiceOverload( + BulkheadService userService, + BulkheadService backgroundService) { + + LOGGER.info("Overloading background service - user service should remain responsive"); + + // Flood background service with tasks (more than its capacity) + for (int i = 1; i <= 10; i++) { + Task task = new Task("Heavy-Background-Job-" + i, TaskType.BACKGROUND_PROCESSING, 2000); + backgroundService.submitTask(task); + } + + // User service should still accept requests + for (int i = 1; i <= 3; i++) { + Task task = new Task("Critical-User-Request-" + i, TaskType.USER_REQUEST, 300); + boolean accepted = userService.submitTask(task); + LOGGER.info("User request {} accepted: {}", i, accepted); + } + } + + /** + * Demonstrates resource isolation by showing independent operation of services. + * + * @param userService the user-facing service + * @param backgroundService the background processing service + */ + private static void demonstrateResourceIsolation( + BulkheadService userService, + BulkheadService backgroundService) { + + LOGGER.info("Demonstrating resource isolation between services"); + LOGGER.info("User Service - Active threads: {}, Queue size: {}", + userService.getActiveThreads(), userService.getQueueSize()); + LOGGER.info("Background Service - Active threads: {}, Queue size: {}", + backgroundService.getActiveThreads(), backgroundService.getQueueSize()); + } +} \ No newline at end of file diff --git a/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BackgroundService.java b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BackgroundService.java new file mode 100644 index 000000000000..7817281326b2 --- /dev/null +++ b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BackgroundService.java @@ -0,0 +1,38 @@ +package com.iluwatar.bulkhead; + +/** + * Service for handling background processing tasks with dedicated resources. + * + *

This service handles non-critical, asynchronous operations that can tolerate + * higher latency. By isolating its resources from user-facing services, failures + * or slowdowns in background processing don't impact critical user operations. + * + *

Example use cases: + *

+ */ +public class BackgroundService extends BulkheadService { + + private static final int DEFAULT_QUEUE_CAPACITY = 20; + + /** + * Creates a BackgroundService with specified thread pool size. + * + * @param maxThreads maximum number of threads for background processing + */ + public BackgroundService(int maxThreads) { + super("BackgroundService", maxThreads, DEFAULT_QUEUE_CAPACITY); + } + + @Override + protected void handleRejectedTask(Task task) { + // For background tasks, we might want to retry later or persist to a queue + super.handleRejectedTask(task); + // Additional background-specific rejection logic could go here + // e.g., persist to database for later retry + } +} diff --git a/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadService.java b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadService.java new file mode 100644 index 000000000000..2400b3fd506c --- /dev/null +++ b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/BulkheadService.java @@ -0,0 +1,143 @@ +package com.iluwatar.bulkhead; + +import java.util.concurrent.ArrayBlockingQueue; +import java.util.concurrent.RejectedExecutionException; +import java.util.concurrent.ThreadPoolExecutor; +import java.util.concurrent.TimeUnit; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +/** + * Abstract base class for services implementing the Bulkhead pattern. + * + *

This class provides resource isolation by maintaining a dedicated thread pool + * for each service. When the thread pool is exhausted, new tasks are rejected quickly + * (fail-fast behavior) rather than blocking or consuming resources from other services. + * + *

Key features: + *

+ */ +public abstract class BulkheadService { + + private static final Logger LOGGER = LoggerFactory.getLogger(BulkheadService.class); + + protected final ThreadPoolExecutor executor; + protected final String serviceName; + private final int maxPoolSize; + private final int queueCapacity; + + /** + * Creates a new BulkheadService with specified thread pool configuration. + * + * @param serviceName name of the service for logging + * @param maxPoolSize maximum number of threads in the pool + * @param queueCapacity maximum number of tasks that can be queued + */ + protected BulkheadService(String serviceName, int maxPoolSize, int queueCapacity) { + this.serviceName = serviceName; + this.maxPoolSize = maxPoolSize; + this.queueCapacity = queueCapacity; + + // Create thread pool with bounded queue + this.executor = new ThreadPoolExecutor( + maxPoolSize, // core pool size + maxPoolSize, // maximum pool size + 60L, // keep alive time + TimeUnit.SECONDS, + new ArrayBlockingQueue<>(queueCapacity), // bounded queue + new ThreadPoolExecutor.AbortPolicy() // reject when full + ); + + LOGGER.info("Created {} with {} threads and queue capacity {}", + serviceName, maxPoolSize, queueCapacity); + } + + /** + * Submits a task for execution. + * Returns false if the task is rejected due to resource exhaustion. + * + * @param task the task to execute + * @return true if task was accepted, false if rejected + */ + public boolean submitTask(Task task) { + try { + executor.execute(() -> processTask(task)); + LOGGER.info("[{}] Task '{}' submitted successfully", serviceName, task.getName()); + return true; + } catch (RejectedExecutionException e) { + LOGGER.warn("[{}] Task '{}' REJECTED - bulkhead is full (fail-fast)", + serviceName, task.getName()); + handleRejectedTask(task); + return false; + } + } + + /** + * Processes the given task. Subclasses can override for custom behavior. + * + * @param task the task to process + */ + protected void processTask(Task task) { + LOGGER.info("[{}] Processing task '{}' (type: {}, duration: {}ms)", + serviceName, task.getName(), task.getType(), task.getDurationMs()); + + try { + // Simulate task processing + Thread.sleep(task.getDurationMs()); + LOGGER.info("[{}] Task '{}' completed successfully", serviceName, task.getName()); + } catch (InterruptedException e) { + LOGGER.error("[{}] Task '{}' interrupted", serviceName, task.getName()); + Thread.currentThread().interrupt(); + } + } + + /** + * Handles a rejected task. Subclasses can override for custom rejection handling. + * + * @param task the rejected task + */ + protected void handleRejectedTask(Task task) { + // Default: just log the rejection + LOGGER.info("[{}] Rejected task '{}' - consider retry or fallback logic", + serviceName, task.getName()); + } + + /** + * Gets the number of currently active threads. + * + * @return number of active threads + */ + public int getActiveThreads() { + return executor.getActiveCount(); + } + + /** + * Gets the current queue size. + * + * @return number of tasks in queue + */ + public int getQueueSize() { + return executor.getQueue().size(); + } + + /** + * Shuts down the thread pool gracefully. + */ + public void shutdown() { + LOGGER.info("[{}] Shutting down thread pool", serviceName); + executor.shutdown(); + try { + if (!executor.awaitTermination(5, TimeUnit.SECONDS)) { + executor.shutdownNow(); + } + } catch (InterruptedException e) { + executor.shutdownNow(); + Thread.currentThread().interrupt(); + } + } +} diff --git a/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Task.java b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Task.java new file mode 100644 index 000000000000..ef20bbdb9e9f --- /dev/null +++ b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/Task.java @@ -0,0 +1,60 @@ +package com.iluwatar.bulkhead; + +/** + * Represents a task to be executed by a bulkhead service. + * + *

Tasks are characterized by their name, type, and expected duration. + * This information is useful for logging, monitoring, and demonstrating + * the bulkhead pattern's behavior under different loads. + */ +public class Task { + + private final String name; + private final TaskType type; + private final long durationMs; + + /** + * Creates a new Task. + * + * @param name unique identifier for the task + * @param type the type of task (user request or background processing) + * @param durationMs expected duration in milliseconds + */ + public Task(String name, TaskType type, long durationMs) { + this.name = name; + this.type = type; + this.durationMs = durationMs; + } + + /** + * Gets the task name. + * + * @return task name + */ + public String getName() { + return name; + } + + /** + * Gets the task type. + * + * @return task type + */ + public TaskType getType() { + return type; + } + + /** + * Gets the expected task duration. + * + * @return duration in milliseconds + */ + public long getDurationMs() { + return durationMs; + } + + @Override + public String toString() { + return "Task{name='" + name + "', type=" + type + ", duration=" + durationMs + "ms}"; + } +} diff --git a/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/TaskType.java b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/TaskType.java new file mode 100644 index 000000000000..7b6d44d1d075 --- /dev/null +++ b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/TaskType.java @@ -0,0 +1,21 @@ +package com.iluwatar.bulkhead; + +/** + * Enumeration of task types in the system. + * + *

Different task types may have different priorities, resource requirements, + * and SLA expectations. The bulkhead pattern allows us to isolate resources + * based on these types. + */ +public enum TaskType { + + /** + * User-facing requests that require low latency and high availability. + */ + USER_REQUEST, + + /** + * Background processing tasks that can tolerate higher latency. + */ + BACKGROUND_PROCESSING +} diff --git a/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/UserService.java b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/UserService.java new file mode 100644 index 000000000000..9fe589880331 --- /dev/null +++ b/microservices-bulkhead/src/main/java/com/iluwatar/bulkhead/UserService.java @@ -0,0 +1,38 @@ +package com.iluwatar.bulkhead; + +/** + * Service for handling user-facing requests with dedicated resources. + * + *

This service represents critical user interactions that require high availability + * and fast response times. By isolating its resources using the Bulkhead pattern, + * it remains responsive even when other services (like background processing) are + * experiencing issues or heavy load. + * + *

Example use cases: + *

+ */ +public class UserService extends BulkheadService { + + private static final int DEFAULT_QUEUE_CAPACITY = 10; + + /** + * Creates a UserService with specified thread pool size. + * + * @param maxThreads maximum number of threads for handling user requests + */ + public UserService(int maxThreads) { + super("UserService", maxThreads, DEFAULT_QUEUE_CAPACITY); + } + + @Override + protected void handleRejectedTask(Task task) { + // For user-facing requests, we might want to return an error to the user + // or implement a fallback mechanism + super.handleRejectedTask(task); + // Additional user-specific rejection logic could go here + } +}