Only check for minimum chunk size after finding a split point #29
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I've found another small performance optimization (up to 5%) for the chunker.
The chunker already skips nearly the complete minimum chunk size and
only processes the last 64 bytes to synchronize the shifting window.
The check whether a chunk already has the minimum chunk size always
happened before checking whether the chunk has found a split point.
The minimum size check is, however, only relevant for the first 64 bytes
used to synchronize the shifting window and never applies later on.
This change flips the order in which both checks are applied. That is
the minimum chunk size is only checked when a split point candidate was
found. As the latter happens rarely this removes one comparison and jump
from the inner loop.
Performance (30 seconds benchtime, 5 repetitions, formatted using
benchstat, benchmark is single threaded):
Old CPU: Intel(R) Xeon(R) CPU E5506 @ 2.13GHz
Chunker-4 269MB/s ± 4% 283MB/s ± 2% +4.97% (p=0.008 n=5+5)
Modern CPU: Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz
Chunker-8 563MB/s ± 0% 563MB/s ± 0% ~ (p=0.111 n=5+4)