Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only check for minimum chunk size after finding a split point #29

Merged
merged 1 commit into from
Jun 14, 2020

Conversation

MichaelEischer
Copy link
Member

I've found another small performance optimization (up to 5%) for the chunker.

The chunker already skips nearly the complete minimum chunk size and
only processes the last 64 bytes to synchronize the shifting window.
The check whether a chunk already has the minimum chunk size always
happened before checking whether the chunk has found a split point.
The minimum size check is, however, only relevant for the first 64 bytes
used to synchronize the shifting window and never applies later on.

This change flips the order in which both checks are applied. That is
the minimum chunk size is only checked when a split point candidate was
found. As the latter happens rarely this removes one comparison and jump
from the inner loop.

Performance (30 seconds benchtime, 5 repetitions, formatted using
benchstat, benchmark is single threaded):
Old CPU: Intel(R) Xeon(R) CPU E5506 @ 2.13GHz
Chunker-4 269MB/s ± 4% 283MB/s ± 2% +4.97% (p=0.008 n=5+5)

Modern CPU: Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz
Chunker-8 563MB/s ± 0% 563MB/s ± 0% ~ (p=0.111 n=5+4)

The chunker already skips nearly the complete minimum chunk size and
only processes the last 64 bytes to synchronize the shifting window.
The check whether a chunk already has the minimum chunk size always
happened before checking whether the chunk has found a split point.
The minimum size check is, however, only relevant for the first 64 bytes
used to synchronize the shifting window and never applies later on.

This change flips the order in which both checks are applied. That is
the minimum chunk size is only checked when a split point candidate was
found. As the latter happens rarely this removes one comparison and jump
from the inner loop.

Performance (30 seconds benchtime, 5 repetitions, formatted using
benchstat, benchmark is single threaded):
Old CPU: Intel(R) Xeon(R) CPU E5506 @ 2.13GHz
Chunker-4  269MB/s ± 4%  283MB/s ± 2%  +4.97%  (p=0.008 n=5+5)

Modern CPU: Intel(R) Xeon(R) CPU E3-1275 v5 @ 3.60GHz
Chunker-8  563MB/s ± 0%  563MB/s ± 0%   ~     (p=0.111 n=5+4)
@MichaelEischer MichaelEischer force-pushed the speedups branch 2 times, most recently from 4760daa to f0753ff Compare June 13, 2020 14:10
@MichaelEischer MichaelEischer marked this pull request as draft June 13, 2020 14:13
@MichaelEischer MichaelEischer marked this pull request as ready for review June 13, 2020 14:15
@fd0 fd0 merged commit 610a379 into restic:master Jun 14, 2020
@fd0
Copy link
Member

fd0 commented Jun 14, 2020

Awesome, thanks!

@MichaelEischer MichaelEischer deleted the speedups branch October 15, 2022 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants