Improve the performance of partitioned step #725
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
New method on JobRepository that enables better performance of SimpleStepExecutionSplitter. By retrieving all step executions at the time of partitioning outside the for loop, the number of calls to the database is reduced. Also updated JSR version to be performant. Deprecated the older step execution methods and cleaned up the javadocs. Stub implementations provided for common support classes https://jira.spring.io/browse/BATCH-2716
Contributor
Author
|
Hi @NaanProphet , After further review of this PR, we had a couple of concerns:
After digging deeper, we thought it should be possible to improve the performance of getting the last step execution for a given job instance using a sql query instead of grabbing all step executions from the database and doing the filtering in memory. This is what we've done in #727 and the results are comparable to those of this PR. Hence, we merged the other PR as it does not introduce the couple of issues mentioned previously. I'm closing this PR. Thank you anyway for your contribution! Best regards, |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR is based on #716. It adds the minor changes requested in the code review.
The performance improvement can be measured using this sample app (See
result-*.mdfiles).Resolves BATCH-2716.