Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade klauspost/compress to v1.14.4 #740

Merged

Conversation

dferstay
Copy link
Contributor

Motivation

This commit updates klauspost/compress to v1.14.4 which provides some
performance improvements for pure go zstd compression and decompression.
For details, see:
https://github.com/klauspost/compress/releases

Results from pulsar/internal/compression/compression_bench_test.go show the speedup.

name                                        old time/op    new time/op     delta
Compression/zstd-pure-go-fastest               967µs ± 0%      907µs ± 1%   -6.21%  (p=0.000 n=8+7)
Compression/zstd-pure-go-fastest-8             968µs ± 1%      779µs ±13%  -19.57%  (p=0.000 n=8+8)
Compression/zstd-pure-go-default              1.52ms ± 1%     1.45ms ± 0%   -4.35%  (p=0.000 n=8+8)
Compression/zstd-pure-go-default-8            1.52ms ± 2%     1.45ms ± 1%   -4.66%  (p=0.000 n=8+8)
Compression/zstd-pure-go-best                 2.67ms ± 1%     2.68ms ± 0%     ~     (p=0.195 n=8+8)
Compression/zstd-pure-go-best-8               2.66ms ± 2%     2.62ms ± 2%     ~     (p=0.054 n=7+8)
Decompression/zstd-pure-go-fastest             411µs ± 0%      352µs ± 0%  -14.26%  (p=0.000 n=7+8)
Decompression/zstd-pure-go-fastest-8           414µs ± 2%      352µs ± 0%  -15.07%  (p=0.001 n=8+6)
Decompression/zstd-pure-go-default             434µs ± 1%      368µs ± 0%  -15.20%  (p=0.000 n=8+7)
Decompression/zstd-pure-go-default-8           424µs ±11%      368µs ± 0%  -13.05%  (p=0.000 n=8+8)
Decompression/zstd-pure-go-best                437µs ± 1%      364µs ± 0%  -16.54%  (p=0.000 n=8+7)
Decompression/zstd-pure-go-best-8              432µs ± 1%      364µs ± 0%  -15.89%  (p=0.000 n=8+8)
CompressionParallel/zstd-pure-go-fastest      1.20ms ± 2%     0.99ms ± 0%  -17.58%  (p=0.000 n=8+7)
CompressionParallel/zstd-pure-go-fastest-8     308µs ± 2%      273µs ± 1%  -11.36%  (p=0.000 n=7+8)
CompressionParallel/zstd-pure-go-default      1.22ms ± 2%     1.01ms ± 0%  -16.87%  (p=0.000 n=8+8)
CompressionParallel/zstd-pure-go-default-8     306µs ± 2%      270µs ± 2%  -11.72%  (p=0.000 n=7+8)
CompressionParallel/zstd-pure-go-best         1.20ms ± 3%     1.00ms ± 0%  -16.41%  (p=0.000 n=8+7)
CompressionParallel/zstd-pure-go-best-8        305µs ± 2%      272µs ± 1%  -10.77%  (p=0.000 n=7+8)

name                                        old speed      new speed       delta
Compression/zstd-pure-go-fastest             104MB/s ± 0%    111MB/s ± 1%   +6.63%  (p=0.000 n=8+7)
Compression/zstd-pure-go-fastest-8           104MB/s ± 1%    130MB/s ±12%  +24.93%  (p=0.000 n=8+8)
Compression/zstd-pure-go-default            66.3MB/s ± 1%   69.4MB/s ± 0%   +4.54%  (p=0.000 n=8+8)
Compression/zstd-pure-go-default-8          66.4MB/s ± 2%   69.6MB/s ± 1%   +4.89%  (p=0.000 n=8+8)
Compression/zstd-pure-go-best               37.7MB/s ± 1%   37.6MB/s ± 0%     ~     (p=0.201 n=8+8)
Compression/zstd-pure-go-best-8             37.9MB/s ± 2%   38.4MB/s ± 2%   +1.36%  (p=0.044 n=7+8)
Decompression/zstd-pure-go-fastest           245MB/s ± 0%    285MB/s ± 0%  +16.63%  (p=0.000 n=7+8)
Decompression/zstd-pure-go-fastest-8         243MB/s ± 1%    286MB/s ± 0%  +17.73%  (p=0.001 n=8+6)
Decompression/zstd-pure-go-default           232MB/s ± 1%    273MB/s ± 0%  +17.92%  (p=0.000 n=8+7)
Decompression/zstd-pure-go-default-8         239MB/s ±12%    273MB/s ± 0%  +14.46%  (p=0.000 n=8+8)
Decompression/zstd-pure-go-best              230MB/s ± 1%    276MB/s ± 0%  +19.82%  (p=0.000 n=8+7)
Decompression/zstd-pure-go-best-8            233MB/s ± 1%    277MB/s ± 0%  +18.89%  (p=0.000 n=8+8)
CompressionParallel/zstd-pure-go-fastest    84.1MB/s ± 2%  102.0MB/s ± 0%  +21.32%  (p=0.000 n=8+7)
CompressionParallel/zstd-pure-go-fastest-8   326MB/s ± 2%    368MB/s ± 1%  +12.80%  (p=0.000 n=7+8)
CompressionParallel/zstd-pure-go-default    82.6MB/s ± 2%   99.3MB/s ± 0%  +20.28%  (p=0.000 n=8+8)
CompressionParallel/zstd-pure-go-default-8   328MB/s ± 2%    372MB/s ± 2%  +13.28%  (p=0.000 n=7+8)
CompressionParallel/zstd-pure-go-best       84.0MB/s ± 3%  100.4MB/s ± 0%  +19.57%  (p=0.001 n=8+6)
CompressionParallel/zstd-pure-go-best-8      330MB/s ± 2%    370MB/s ± 1%  +12.05%  (p=0.000 n=7+8)

Modifications

The version of klauspost/compress has been updated to tag v1.14.4

Verifying this change

This change is already covered by existing tests, such as:.

  • pulsar/internal/compression/compression_test.go
  • pulsar/internal/compression/compression_bench_test.go

Does this pull request potentially affect one of the following parts:

  • Dependencies (does it add or upgrade a dependency): Yes, it moves to a newer version of github.com/klauspost/compress
  • The public API: No
  • The schema: No
  • The default values of configurations: No
  • The wire protocol: No

Documentation

  • Does this pull request introduce a new feature? No

This commit updates klauspost/compress to v1.14.4 which provides some
performance improvements for pure go zstd compression and decompression.
For details, see:
https://github.com/klauspost/compress/releases

Signed-off-by: Daniel Ferstay <dferstay@splunk.com>
@merlimat merlimat added this to the v0.9.0 milestone Feb 26, 2022
@merlimat merlimat merged commit 411a0cd into apache:master Feb 26, 2022
@freeznet freeznet modified the milestones: v0.9.0, v0.8.1 Mar 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants