Tech debt: Refactor batch module to eliminate code-duplication #2302
Description
Why is this needed?
Historically, the Batch utility supported SQS event sources. Today, it supports more than just SQS. It also supports Kinesis and DynamoDB stream event sources.
These additionally event sources were added as a separate "handlers" in:
- https://github.com/aws-powertools/powertools-lambda-java/blob/main/powertools-batch/src/main/java/software/amazon/lambda/powertools/batch/handler/DynamoDbBatchMessageHandler.java
- https://github.com/aws-powertools/powertools-lambda-java/blob/main/powertools-batch/src/main/java/software/amazon/lambda/powertools/batch/handler/KinesisStreamsBatchMessageHandler.java
- https://github.com/aws-powertools/powertools-lambda-java/blob/main/powertools-batch/src/main/java/software/amazon/lambda/powertools/batch/handler/SqsBatchMessageHandler.java
While this approach works, it also introduces a lot of code duplication since the whole code is repeated for different method overloads such as for processBatchInParallel and also in between the different handlers.
This also leads regularly to Sonarcube findings when sending PRs involving these handlers.
- Example: improv(batch): Propagate trace entity to worker threads during parallel processing. #2300
- Sonar finding: https://sonarcloud.io/component_measures?id=aws-powertools_powertools-lambda-java&pullRequest=2300&metric=new_duplicated_lines_density&view=list
Which area does this relate to?
Batch processing
Suggestion
This issue suggest to refactor the batch module in a way to remove this redundant logic since the majority of code is shared. We should find a way to separate the orchestration logic of the (parallel) batch processing from the event source specific responses.
Anyone interested in working on this, please feel free to propose refactoring suggestions.
Acknowledgment
- This request meets Powertools for AWS Lambda (Python) Tenets
- Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, TypeScript, and .NET