Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.Sign up
Performance improvement for partitions with a large number of steps [BATCH-2716] #891
Each time partitions are created,
The patched approach uses a new method on
The performance improvement is easily an order of magnitude better (26x from initial benchmarks, attached) and works well for jobs with > 1000 steps.
Patched source is attached and also available on https://github.com/NaanProphet/spring-batch-large-step-perf-fix
Backported to: 4.2.0.RC1
Mahmoud Ben Hassine commented
Krishna Bhamidipati That's aweswome! Thank you for this analysis and benchmarks! This issue has been planned for v4.2 for which the theme is "performance ++".
I checked the content of the attached zip file and there are some good ideas in there. Can you please open a PR on Github with those changes? Many thanks upfront.